[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:04 dist-test-slave-7f01 krb5kdc[9157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:04 dist-test-slave-7f01 krb5kdc[9157](info): set up 2 sockets
Aug 11 10:12:04 dist-test-slave-7f01 krb5kdc[9157](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 10:12:05.533023 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.440s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:05 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:05.658288 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42291
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:05.746214 9179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:05.746419 9179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:05.746441 9179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:05.747625 9179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:05.747673 9179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:05.747687 9179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:05.747761 9179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:05.747781 9179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:05.749469 9179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:05.749843 9179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:05.750109 9179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:12:05.756799 9179 server_base.cc:1034] running on GCE node
W20260811 10:12:05.758129 9186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:05.758041 9188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:05.758183 9184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:05.758726 9179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:05.759258 9179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:05.760437 9179 hybrid_clock.cc:648] HybridClock initialized: now 1786443125760422 us; error 30 us; skew 500 ppm
Aug 11 10:12:05 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443125, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:05.763132 9179 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:05.763820 9179 webserver.cc:460] Webserver started at http://127.8.238.62:38705/ using document root <none> and password file <none>
I20260811 10:12:05.764146 9179 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:05.764226 9179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:05.764442 9179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:05.765767 9179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "04e8505f644b42a39b9c58888ba1cdf3"
format_stamp: "Formatted at 2026-08-11 10:12:05 on dist-test-slave-7f01"
server_key: "de9e01098220980b8c0eade2d2d6eaaa"
server_key_iv: "14ab430a853b8db23d4bf7b249218d9d"
server_key_version: "encryptionkey@0"
I20260811 10:12:05.766352 9179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "04e8505f644b42a39b9c58888ba1cdf3"
format_stamp: "Formatted at 2026-08-11 10:12:05 on dist-test-slave-7f01"
server_key: "de9e01098220980b8c0eade2d2d6eaaa"
server_key_iv: "14ab430a853b8db23d4bf7b249218d9d"
server_key_version: "encryptionkey@0"
I20260811 10:12:05.768270 9179 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260811 10:12:05.769332 9202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:05.769573 9179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:05.769699 9179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "04e8505f644b42a39b9c58888ba1cdf3"
format_stamp: "Formatted at 2026-08-11 10:12:05 on dist-test-slave-7f01"
server_key: "de9e01098220980b8c0eade2d2d6eaaa"
server_key_iv: "14ab430a853b8db23d4bf7b249218d9d"
server_key_version: "encryptionkey@0"
I20260811 10:12:05.769843 9179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:05.788085 9179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:05.790241 9179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:05.790446 9179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:05.797503 9179 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:42291
I20260811 10:12:05.797686 9305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:42291 every 8 connection(s)
I20260811 10:12:05.798183 9179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:05.799914 9308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:05.803622 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9179
I20260811 10:12:05.803696 9308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3: Bootstrap starting.
I20260811 10:12:05.803745 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:05.804375 9144 external_mini_cluster.cc:1428] Setting key f4b42b23a80ab221a62487c8f8fcc080
I20260811 10:12:05.804996 9308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:05.805840 9308 log.cc:824] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:05.806957 9308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3: No bootstrap required, opened a new log
I20260811 10:12:05.810318 9308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } }
I20260811 10:12:05.810536 9308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:05.810647 9308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04e8505f644b42a39b9c58888ba1cdf3, State: Initialized, Role: FOLLOWER
I20260811 10:12:05.810869 9308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } }
I20260811 10:12:05.811017 9308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:05.811070 9308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:05.811126 9308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:05.812292 9308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } }
I20260811 10:12:05.812510 9308 leader_election.cc:302] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04e8505f644b42a39b9c58888ba1cdf3; no voters:
I20260811 10:12:05.812740 9308 leader_election.cc:288] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:05.812841 9314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:05.813112 9314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [term 1 LEADER]: Becoming Leader. State: Replica: 04e8505f644b42a39b9c58888ba1cdf3, State: Running, Role: LEADER
I20260811 10:12:05.813271 9308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:05.813259 9314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } }
I20260811 10:12:05.814370 9317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04e8505f644b42a39b9c58888ba1cdf3. Latest consensus state: current_term: 1 leader_uuid: "04e8505f644b42a39b9c58888ba1cdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } } }
I20260811 10:12:05.814549 9317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:05.814635 9315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04e8505f644b42a39b9c58888ba1cdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04e8505f644b42a39b9c58888ba1cdf3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42291 } } }
I20260811 10:12:05.814745 9315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:05.815042 9335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:05.816362 9335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:05.820340 9335 catalog_manager.cc:1347] Generated new cluster ID: e215c9b8da954328a19cc25f2d0d2118
I20260811 10:12:05.820416 9335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:12:05 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:05.829519 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:05.805660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.8.238.62:42291)
0811 10:12:05.806024 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:05.806032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:05.806409 (+ 377us) server_negotiation.cc:407] Connection header received
0811 10:12:05.810292 (+ 3883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:05.810298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:05.810390 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:05.810534 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:05.812949 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:05.815376 (+ 2427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:05.817603 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:05.818296 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:05.818459 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:05.824186 (+ 5727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:05.824191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:05.824191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:05.824203 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:05.826962 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:05.827520 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:05.827524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:05.827524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:05.827708 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:05.828072 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:05.828073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:05.828073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:05.828556 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:05.828875 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:05.828972 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 10:12:05.829160 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":171,"threads_started":1}
I20260811 10:12:05.831288 9335 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:05.831862 9335 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:05.834221 9335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04e8505f644b42a39b9c58888ba1cdf3: Generated new TSK 0
I20260811 10:12:05.834400 9335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:05.875002 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:06.004875 9358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:06.005084 9358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:06.005110 9358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:06.006323 9358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:06.006369 9358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:06.006397 9358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:06.008018 9358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:06.008394 9358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:06.008715 9358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:06.016009 9366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.016029 9368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.016364 9371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:06.016752 9358 server_base.cc:1034] running on GCE node
I20260811 10:12:06.016964 9358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:06.017220 9358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:06.018392 9358 hybrid_clock.cc:648] HybridClock initialized: now 1786443126018367 us; error 37 us; skew 500 ppm
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:06.021850 9358 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:06.022339 9358 webserver.cc:460] Webserver started at http://127.8.238.1:37825/ using document root <none> and password file <none>
I20260811 10:12:06.022598 9358 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:06.022671 9358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:06.022832 9358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:06.024102 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "bfdd606fb85942358ff49257b40b3d92"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "a78c7cf3afb0a5d7e921d8b5398e447c"
server_key_iv: "fe498c648d885c75d0b3148d0603ae1a"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.024514 9358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "bfdd606fb85942358ff49257b40b3d92"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "a78c7cf3afb0a5d7e921d8b5398e447c"
server_key_iv: "fe498c648d885c75d0b3148d0603ae1a"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.026031 9358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:06.026949 9384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.027146 9358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:06.027258 9358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "bfdd606fb85942358ff49257b40b3d92"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "a78c7cf3afb0a5d7e921d8b5398e447c"
server_key_iv: "fe498c648d885c75d0b3148d0603ae1a"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.027354 9358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:06.045197 9358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:06.047878 9358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:06.048061 9358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:06.048454 9358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:06.048897 9358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:06.049006 9358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.049052 9358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:06.049082 9358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.060485 9358 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:42037
I20260811 10:12:06.060559 9583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:42037 every 8 connection(s)
I20260811 10:12:06.060984 9358 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:06.067803 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9358
I20260811 10:12:06.067950 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:06.068120 9144 external_mini_cluster.cc:1428] Setting key 8da656d9859a8ffdc30bf29f13a46e56
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:06.072037 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.062076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:40879 (local address 127.8.238.62:42291)
0811 10:12:06.062144 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:06.062146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:06.062390 (+ 244us) server_negotiation.cc:407] Connection header received
0811 10:12:06.063014 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:06.063015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:06.063030 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:06.063104 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:06.064413 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.064932 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.065976 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.066188 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.066245 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.068716 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:06.068717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:06.068717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:06.068724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:06.070203 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.070784 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.070786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.070786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.070850 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.071190 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.071191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.071192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.071356 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:06.071437 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:06.071721 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 10:12:06.071905 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:12:06.072117 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.062058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42291 (local address 127.8.238.1:40879)
0811 10:12:06.062273 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:06.062310 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:06.062911 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:06.063194 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:06.063198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:06.064053 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:06.064295 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.064301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.065022 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.065023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.065888 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.065891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.066266 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.066267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.066385 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.066977 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:06.066986 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:06.068628 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:06.070297 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.070306 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.070308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.070671 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.070937 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.070938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.070939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.071092 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.071439 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:06.071445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:06.071581 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 10:12:06.071667 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:12:06.071887 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260811 10:12:06.072772 9584 heartbeater.cc:347] Connected to a master server at 127.8.238.62:42291
I20260811 10:12:06.072902 9584 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:06.073107 9584 heartbeater.cc:511] Master 127.8.238.62:42291 requested a full tablet report, sending...
I20260811 10:12:06.074103 9239 ts_manager.cc:194] Registered new tserver with Master: bfdd606fb85942358ff49257b40b3d92 (127.8.238.1:42037)
I20260811 10:12:06.075274 9239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:40879
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:06.098853 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:06.191908 9604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:06.192165 9604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:06.192206 9604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:06.193382 9604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:06.193456 9604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:06.193501 9604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:06.195050 9604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:06.195320 9604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:06.195561 9604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:06.201377 9623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.201406 9616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.201550 9621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:06.201959 9604 server_base.cc:1034] running on GCE node
I20260811 10:12:06.202178 9604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:06.202419 9604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:06.203586 9604 hybrid_clock.cc:648] HybridClock initialized: now 1786443126203573 us; error 30 us; skew 500 ppm
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:06.206471 9604 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:06.206974 9604 webserver.cc:460] Webserver started at http://127.8.238.2:43357/ using document root <none> and password file <none>
I20260811 10:12:06.207185 9604 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:06.207226 9604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:06.207324 9604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:06.208462 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "b2a562f7b4f046d48ae3fef0eac1a086"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "1382a9d5e1f1a4f4d7b4501a00c33d72"
server_key_iv: "d14e099701ae16e25b6523828e4a1c49"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.208840 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "b2a562f7b4f046d48ae3fef0eac1a086"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "1382a9d5e1f1a4f4d7b4501a00c33d72"
server_key_iv: "d14e099701ae16e25b6523828e4a1c49"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.210237 9604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:06.210960 9638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.211160 9604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:06.211243 9604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "b2a562f7b4f046d48ae3fef0eac1a086"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "1382a9d5e1f1a4f4d7b4501a00c33d72"
server_key_iv: "d14e099701ae16e25b6523828e4a1c49"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.211294 9604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:06.228412 9604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:06.231079 9604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:06.231235 9604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:06.231591 9604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:06.231977 9604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:06.232038 9604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.232074 9604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:06.232123 9604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.244171 9604 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:34283
I20260811 10:12:06.244287 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:34283 every 8 connection(s)
I20260811 10:12:06.244609 9604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:06.244937 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9604
I20260811 10:12:06.245070 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:06.245271 9144 external_mini_cluster.cc:1428] Setting key 39a883ffcbdb8edefd9e7a302ae91758
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
I20260811 10:12:06.256842 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.245581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:34935 (local address 127.8.238.62:42291)
0811 10:12:06.245636 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:06.245638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:06.245985 (+ 347us) server_negotiation.cc:407] Connection header received
0811 10:12:06.246984 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:06.246986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:06.246998 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:06.247045 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:06.248554 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.249357 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.250444 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.250789 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.250878 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.253576 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:06.253578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:06.253578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:06.253587 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:06.255376 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.255845 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.255847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.255847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.255914 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.256220 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.256221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.256221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.256377 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:06.256455 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:06.256698 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 10:12:06.256773 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:06.257084 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.245561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42291 (local address 127.8.238.2:34935)
0811 10:12:06.245838 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:06.245887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:06.246858 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:06.247120 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:06.247124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:06.248053 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:06.248450 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.248456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.249457 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.249458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.250338 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.250342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.250896 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.250897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.251098 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.251847 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:06.251858 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:06.253477 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:06.255468 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.255475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.255477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.255701 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.255993 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.255993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.255993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.256146 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.256448 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:06.256454 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:06.256570 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 10:12:06.256646 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:12:06.256770 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":154,"threads_started":1}
I20260811 10:12:06.257655 9841 heartbeater.cc:347] Connected to a master server at 127.8.238.62:42291
I20260811 10:12:06.257776 9841 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:06.257972 9841 heartbeater.cc:511] Master 127.8.238.62:42291 requested a full tablet report, sending...
I20260811 10:12:06.258531 9239 ts_manager.cc:194] Registered new tserver with Master: b2a562f7b4f046d48ae3fef0eac1a086 (127.8.238.2:34283)
I20260811 10:12:06.258971 9239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:34935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:06.275360 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:06.382351 9846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:06.382591 9846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:06.382633 9846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:06.383849 9846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:06.383971 9846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:06.384083 9846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:06.385787 9846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:42291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:06.386062 9846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:06.386301 9846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:06.391841 9857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.391935 9860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:06.391963 9856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:06.392310 9846 server_base.cc:1034] running on GCE node
I20260811 10:12:06.392514 9846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:06.392710 9846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:06.393877 9846 hybrid_clock.cc:648] HybridClock initialized: now 1786443126393865 us; error 30 us; skew 500 ppm
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:06.396693 9846 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:06.397292 9846 webserver.cc:460] Webserver started at http://127.8.238.3:34179/ using document root <none> and password file <none>
I20260811 10:12:06.397547 9846 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:06.397594 9846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:06.397687 9846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:06.398699 9846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "6db937664422411da0e9d71235246880"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "96dd9daa6b10432268e5e13bfdf0c866"
server_key_iv: "cc50f56828cf2dddfd6bd70cd4cdd56e"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.399025 9846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "6db937664422411da0e9d71235246880"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "96dd9daa6b10432268e5e13bfdf0c866"
server_key_iv: "cc50f56828cf2dddfd6bd70cd4cdd56e"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.400624 9846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 10:12:06.401499 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.401744 9846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:06.401861 9846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "6db937664422411da0e9d71235246880"
format_stamp: "Formatted at 2026-08-11 10:12:06 on dist-test-slave-7f01"
server_key: "96dd9daa6b10432268e5e13bfdf0c866"
server_key_iv: "cc50f56828cf2dddfd6bd70cd4cdd56e"
server_key_version: "encryptionkey@0"
I20260811 10:12:06.401954 9846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:06.413663 9846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:06.416127 9846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:06.416281 9846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:06.416524 9846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:06.416846 9846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:06.416900 9846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.416944 9846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:06.416972 9846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:06.425606 9846 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:38759
I20260811 10:12:06.425648 10077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:38759 every 8 connection(s)
I20260811 10:12:06.425987 9846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:06.431375 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9846
I20260811 10:12:06.431478 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:06.431651 9144 external_mini_cluster.cc:1428] Setting key bcf7b780413a690842cfcb11d7dae24c
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:06.435999 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.426857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:40355 (local address 127.8.238.62:42291)
0811 10:12:06.426930 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:06.426933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:06.427146 (+ 213us) server_negotiation.cc:407] Connection header received
0811 10:12:06.427781 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:06.427783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:06.427798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:06.427850 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:06.428857 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.429379 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.430472 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.430684 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.430769 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.433500 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:06.433501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:06.433501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:06.433507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:06.434714 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.435100 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.435101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.435102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.435157 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.435380 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.435381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.435381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.435491 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:06.435543 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:06.435848 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 10:12:06.435915 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:12:06.436250 10088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.426853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42291 (local address 127.8.238.3:40355)
0811 10:12:06.427019 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:06.427055 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:06.427632 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:06.427932 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:06.427936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:06.428549 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:06.428748 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.428753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.429489 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.429490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.430373 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:06.430375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.430784 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.430785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:06.430947 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.431470 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:06.431482 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:06.433419 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:06.434785 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.434792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.434794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.435033 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.435219 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:06.435220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:06.435220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:06.435323 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:06.435571 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:06.435578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:06.435703 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 10:12:06.435827 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:12:06.435978 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260811 10:12:06.436735 10082 heartbeater.cc:347] Connected to a master server at 127.8.238.62:42291
I20260811 10:12:06.436865 10082 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:06.437049 10082 heartbeater.cc:511] Master 127.8.238.62:42291 requested a full tablet report, sending...
I20260811 10:12:06.437479 9239 ts_manager.cc:194] Registered new tserver with Master: 6db937664422411da0e9d71235246880 (127.8.238.3:38759)
I20260811 10:12:06.437820 9239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:40355
I20260811 10:12:06.444042 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:06.470857 10098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.462845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.8.238.1:42037)
0811 10:12:06.463062 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:06.463068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:06.463080 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:12:06.463111 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:06.463113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:06.463128 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:06.463205 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:06.463850 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.464614 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.465366 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.465698 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.465767 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.468237 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:06.468245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:06.468245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:06.468252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:06.469687 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.470038 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.470040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.470041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.470102 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.470375 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.470376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.470376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.470513 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:06.470633 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:06.470698 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 10:12:06.470764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
W20260811 10:12:06.471371 9433 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56428
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[9157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:06.496217 10098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:06.488758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.8.238.1:42037)
0811 10:12:06.488836 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:06.488838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:06.488897 (+ 59us) server_negotiation.cc:407] Connection header received
0811 10:12:06.489002 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:06.489003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:06.489025 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:06.489094 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:06.489697 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.490216 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.490990 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:06.491213 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:06.491303 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:06.493607 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:06.493609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:06.493609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:06.493616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:06.494998 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.495423 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.495425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.495425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.495488 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:06.495674 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:06.495674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:06.495675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:06.495866 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:06.495971 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:06.496070 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 10:12:06.496145 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:12:06.496958 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9358
I20260811 10:12:06.515183 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9604
I20260811 10:12:06.526355 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9846
I20260811 10:12:06.535033 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9179
2026-08-11T10:12:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2476 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[10113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[10113](info): set up 2 sockets
Aug 11 10:12:06 dist-test-slave-7f01 krb5kdc[10113](info): commencing operation
krb5kdc: starting...
W20260811 10:12:07.837529 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:07 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:07.967166 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34691
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:08.052284 10147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:08.052520 10147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:08.052556 10147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:08.053679 10147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:08.053746 10147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:08.053773 10147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:08.053804 10147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:08.053826 10147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:08.055269 10147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:08.055543 10147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:08.055809 10147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:08.060606 10156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.060618 10154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.060618 10160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:08.060983 10147 server_base.cc:1034] running on GCE node
I20260811 10:12:08.061249 10147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:08.061586 10147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:08.062767 10147 hybrid_clock.cc:648] HybridClock initialized: now 1786443128062740 us; error 52 us; skew 500 ppm
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:08.065322 10147 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:08.065738 10147 webserver.cc:460] Webserver started at http://127.8.238.62:41571/ using document root <none> and password file <none>
I20260811 10:12:08.065991 10147 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:08.066062 10147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:08.066208 10147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:08.067265 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "4fdee2cc963e4947a5447952e3ff713d"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "5e37534b6abd7908fb165945e62a1439"
server_key_iv: "3e1c866c902d0c395a7aa3f6bb17ddee"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.067638 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "4fdee2cc963e4947a5447952e3ff713d"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "5e37534b6abd7908fb165945e62a1439"
server_key_iv: "3e1c866c902d0c395a7aa3f6bb17ddee"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.069027 10147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:08.069743 10175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.069937 10147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:08.070051 10147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "4fdee2cc963e4947a5447952e3ff713d"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "5e37534b6abd7908fb165945e62a1439"
server_key_iv: "3e1c866c902d0c395a7aa3f6bb17ddee"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.070127 10147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:08.081223 10147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:08.083151 10147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:08.083314 10147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:08.087628 10147 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:34691
I20260811 10:12:08.087666 10278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:34691 every 8 connection(s)
I20260811 10:12:08.088063 10147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:08.089073 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:08.091364 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d: Bootstrap starting.
I20260811 10:12:08.092027 10280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:08.092259 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10147
I20260811 10:12:08.092355 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:08.092458 10280 log.cc:824] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:08.092536 9144 external_mini_cluster.cc:1428] Setting key 741d796140975322d13c736fcc003e13
I20260811 10:12:08.093326 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d: No bootstrap required, opened a new log
I20260811 10:12:08.095705 10280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } }
I20260811 10:12:08.095940 10280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:08.095993 10280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fdee2cc963e4947a5447952e3ff713d, State: Initialized, Role: FOLLOWER
I20260811 10:12:08.096122 10280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } }
I20260811 10:12:08.096223 10280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:08.096269 10280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:08.096319 10280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:08.096865 10280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } }
I20260811 10:12:08.097005 10280 leader_election.cc:302] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fdee2cc963e4947a5447952e3ff713d; no voters:
I20260811 10:12:08.097189 10280 leader_election.cc:288] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:08.097239 10287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:08.097472 10280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:08.097682 10287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [term 1 LEADER]: Becoming Leader. State: Replica: 4fdee2cc963e4947a5447952e3ff713d, State: Running, Role: LEADER
I20260811 10:12:08.097774 10287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } }
I20260811 10:12:08.098115 10289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fdee2cc963e4947a5447952e3ff713d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } } }
I20260811 10:12:08.098199 10289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:08.098389 10293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fdee2cc963e4947a5447952e3ff713d. Latest consensus state: current_term: 1 leader_uuid: "4fdee2cc963e4947a5447952e3ff713d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fdee2cc963e4947a5447952e3ff713d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34691 } } }
I20260811 10:12:08.098464 10293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:08.098595 10306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:08.099269 10306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:08.101482 10306 catalog_manager.cc:1347] Generated new cluster ID: 373c8c125ffa4cdfb5bd22942befc8b4
I20260811 10:12:08.101540 10306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:08.104691 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.093038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.8.238.62:34691)
0811 10:12:08.093253 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.093263 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.093305 (+ 42us) server_negotiation.cc:407] Connection header received
0811 10:12:08.093780 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.093793 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.093839 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.093981 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.094643 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.095535 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.096549 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.096968 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.097089 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.100166 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.100170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.100171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.100183 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.102503 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.102988 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.102991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.102992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.103115 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.103386 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.103387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.103388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.103727 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.103968 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.104289 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.104456 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260811 10:12:08.134399 10306 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:08.135008 10306 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:08.146237 10306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fdee2cc963e4947a5447952e3ff713d: Generated new TSK 0
I20260811 10:12:08.146425 10306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:08.179970 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:08.281778 10328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:08.281993 10328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:08.282018 10328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:08.283119 10328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:08.283154 10328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:08.283179 10328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:08.284602 10328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:08.284783 10328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:08.284950 10328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:08.290025 10338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:08.290118 10328 server_base.cc:1034] running on GCE node
W20260811 10:12:08.290041 10342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.290211 10340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:08.290462 10328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:08.290670 10328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:08.291818 10328 hybrid_clock.cc:648] HybridClock initialized: now 1786443128291798 us; error 43 us; skew 500 ppm
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:08.294502 10328 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:08.295053 10328 webserver.cc:460] Webserver started at http://127.8.238.1:40199/ using document root <none> and password file <none>
I20260811 10:12:08.295281 10328 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:08.295327 10328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:08.295461 10328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:08.296546 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "7ac9c888add0456db94bd876a701bd5a"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "4ec3c97fcc15c6fc48f5fd0a7ac512f3"
server_key_iv: "0a5f4b363f49918295af545d4542ecfd"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.296934 10328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "7ac9c888add0456db94bd876a701bd5a"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "4ec3c97fcc15c6fc48f5fd0a7ac512f3"
server_key_iv: "0a5f4b363f49918295af545d4542ecfd"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.298343 10328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:08.299160 10357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.299521 10328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:08.299630 10328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "7ac9c888add0456db94bd876a701bd5a"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "4ec3c97fcc15c6fc48f5fd0a7ac512f3"
server_key_iv: "0a5f4b363f49918295af545d4542ecfd"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.299717 10328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:08.315925 10328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:08.317925 10328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:08.318092 10328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:08.318329 10328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:08.318641 10328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:08.318697 10328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.318738 10328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:08.318766 10328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.325160 10328 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34605
I20260811 10:12:08.325217 10542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34605 every 8 connection(s)
I20260811 10:12:08.325512 10328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:08.325896 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10328
I20260811 10:12:08.326001 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:08.326196 9144 external_mini_cluster.cc:1428] Setting key 64e9e355e63fecd662dfd72050ef38d9
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:08.336514 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.326281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:41819 (local address 127.8.238.62:34691)
0811 10:12:08.326477 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.326479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.326577 (+ 98us) server_negotiation.cc:407] Connection header received
0811 10:12:08.327980 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.327982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.328002 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.328058 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.328944 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.329482 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.330483 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.330706 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.330794 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.333319 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.333321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.333322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.333330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.335035 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.335434 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.335436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.335437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.335512 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.335798 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.335800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.335800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.335964 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.336045 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.336310 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.336405 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260811 10:12:08.336531 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.326250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34691 (local address 127.8.238.1:41819)
0811 10:12:08.326442 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.326484 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.327865 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.328141 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.328145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.328552 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:08.328830 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.328837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.329571 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.329572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.330382 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.330384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.330794 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.330795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.330940 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.331497 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:08.331507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:08.333220 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:08.335122 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.335129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.335131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.335356 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.335591 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.335592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.335593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.335697 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.336042 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:08.336048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:08.336166 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.336252 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.336372 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260811 10:12:08.337071 10543 heartbeater.cc:347] Connected to a master server at 127.8.238.62:34691
I20260811 10:12:08.337178 10543 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:08.337396 10543 heartbeater.cc:511] Master 127.8.238.62:34691 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
I20260811 10:12:08.337972 10211 ts_manager.cc:194] Registered new tserver with Master: 7ac9c888add0456db94bd876a701bd5a (127.8.238.1:34605)
I20260811 10:12:08.338599 10211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:41819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:08.357179 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:08.439548 10551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:08.439805 10551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:08.439891 10551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:08.441099 10551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:08.441165 10551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:08.441213 10551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:08.442777 10551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:08.443051 10551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:08.443276 10551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:08.448143 10569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.448180 10564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.448143 10563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:08.448457 10551 server_base.cc:1034] running on GCE node
I20260811 10:12:08.448710 10551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:08.448909 10551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:08.450080 10551 hybrid_clock.cc:648] HybridClock initialized: now 1786443128450068 us; error 44 us; skew 500 ppm
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:08.452651 10551 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:08.453181 10551 webserver.cc:460] Webserver started at http://127.8.238.2:36175/ using document root <none> and password file <none>
I20260811 10:12:08.453440 10551 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:08.453510 10551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:08.453658 10551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:08.454659 10551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "46cd6fe251c742fdbacc146742398e85"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "dd3c60e39aabe80a7d7a9549c3c9bb99"
server_key_iv: "ebb97657d4c4d8a50076770668a4512a"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.455027 10551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "46cd6fe251c742fdbacc146742398e85"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "dd3c60e39aabe80a7d7a9549c3c9bb99"
server_key_iv: "ebb97657d4c4d8a50076770668a4512a"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.456431 10551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:12:08.457123 10579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.457314 10551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:08.457429 10551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "46cd6fe251c742fdbacc146742398e85"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "dd3c60e39aabe80a7d7a9549c3c9bb99"
server_key_iv: "ebb97657d4c4d8a50076770668a4512a"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.457504 10551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:08.467510 10551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:08.469502 10551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:08.469669 10551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:08.469888 10551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:08.470186 10551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:08.470240 10551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.470273 10551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:08.470301 10551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.476806 10551 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:39641
I20260811 10:12:08.476855 10773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:39641 every 8 connection(s)
I20260811 10:12:08.477200 10551 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:08.482326 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10551
I20260811 10:12:08.482455 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:08.482636 9144 external_mini_cluster.cc:1428] Setting key f7164ac9b081c2205750bf63e9e391b3
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:08.487264 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.477989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:47109 (local address 127.8.238.62:34691)
0811 10:12:08.478058 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.478060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.478306 (+ 246us) server_negotiation.cc:407] Connection header received
0811 10:12:08.479219 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.479219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.479231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.479284 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.479961 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.480504 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.481492 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.481681 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.481737 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.484249 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.484251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.484252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.484259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.485743 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.486248 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.486249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.486250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.486307 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.486602 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.486603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.486603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.486722 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.486814 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.487091 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.487174 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:08.487450 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.477965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34691 (local address 127.8.238.2:47109)
0811 10:12:08.478147 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.478214 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.479126 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.479355 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.479358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.479603 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:08.479870 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.479875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.480600 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.480601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.481411 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.481414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.481759 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.481760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.481872 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.482460 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:08.482470 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:08.484152 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:08.485837 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.485845 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.485848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.486168 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.486407 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.486413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.486413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.486522 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.486803 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:08.486811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:08.486947 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.487052 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.487214 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260811 10:12:08.488008 10774 heartbeater.cc:347] Connected to a master server at 127.8.238.62:34691
I20260811 10:12:08.488127 10774 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:08.488334 10774 heartbeater.cc:511] Master 127.8.238.62:34691 requested a full tablet report, sending...
I20260811 10:12:08.488754 10211 ts_manager.cc:194] Registered new tserver with Master: 46cd6fe251c742fdbacc146742398e85 (127.8.238.2:39641)
I20260811 10:12:08.489320 10211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:47109
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:08.511627 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:08.595935 10780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:08.596134 10780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:08.596160 10780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:08.597412 10780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:08.597453 10780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:08.597481 10780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:08.599167 10780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:34691
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:08.599444 10780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:08.599705 10780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:08.604951 10792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.604997 10788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:08.604990 10787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:08.605298 10780 server_base.cc:1034] running on GCE node
I20260811 10:12:08.605522 10780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:08.605772 10780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:08.606959 10780 hybrid_clock.cc:648] HybridClock initialized: now 1786443128606954 us; error 39 us; skew 500 ppm
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:08.609740 10780 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:08.610198 10780 webserver.cc:460] Webserver started at http://127.8.238.3:35267/ using document root <none> and password file <none>
I20260811 10:12:08.610415 10780 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:08.610457 10780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:08.610543 10780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:08.611547 10780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "d26a6d05dc454f39b68803fe90b7c972"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "26f75477d64fd55a35c4841d0a605df2"
server_key_iv: "6fa5ed819379300e0a6d7f357006d462"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.611974 10780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "d26a6d05dc454f39b68803fe90b7c972"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "26f75477d64fd55a35c4841d0a605df2"
server_key_iv: "6fa5ed819379300e0a6d7f357006d462"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.613310 10780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:08.613987 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.614187 10780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:08.614279 10780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "d26a6d05dc454f39b68803fe90b7c972"
format_stamp: "Formatted at 2026-08-11 10:12:08 on dist-test-slave-7f01"
server_key: "26f75477d64fd55a35c4841d0a605df2"
server_key_iv: "6fa5ed819379300e0a6d7f357006d462"
server_key_version: "encryptionkey@0"
I20260811 10:12:08.614332 10780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:08.629007 10780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:08.630895 10780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:08.631031 10780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:08.631220 10780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:08.631556 10780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:08.631592 10780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.631637 10780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:08.631664 10780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:08.638118 10780 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:34795
I20260811 10:12:08.638221 11003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:34795 every 8 connection(s)
I20260811 10:12:08.638538 10780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:08.646996 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10780
I20260811 10:12:08.647079 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:08.647246 9144 external_mini_cluster.cc:1428] Setting key 0cdd7e5dfc65ff701feeae37204a77d8
I20260811 10:12:08.648869 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.639362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:34395 (local address 127.8.238.62:34691)
0811 10:12:08.639412 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.639414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.639664 (+ 250us) server_negotiation.cc:407] Connection header received
0811 10:12:08.640916 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.640917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.640932 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.640985 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.641651 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.642178 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.643209 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.643418 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.643495 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.645685 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.645687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.645687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.645694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.647411 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.647969 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.647970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.647971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.648044 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.648355 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.648356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.648356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.648477 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.648529 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.648744 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.648806 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:12:08.649050 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.639312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34691 (local address 127.8.238.3:34395)
0811 10:12:08.639534 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.639572 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.640782 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.641068 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.641073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.641347 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:08.641547 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.641551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.642281 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.642282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.643110 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.643112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.643516 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.643517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.643647 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.644133 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:08.644142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:08.645598 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:08.647517 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.647525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.647527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.647887 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.648113 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:08.648114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:08.648114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:08.648273 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:08.648532 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:08.648538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:08.648652 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.648726 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.648843 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260811 10:12:08.649560 11004 heartbeater.cc:347] Connected to a master server at 127.8.238.62:34691
I20260811 10:12:08.649670 11004 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:08.649847 11004 heartbeater.cc:511] Master 127.8.238.62:34691 requested a full tablet report, sending...
I20260811 10:12:08.650243 10211 ts_manager.cc:194] Registered new tserver with Master: d26a6d05dc454f39b68803fe90b7c972 (127.8.238.3:34795)
I20260811 10:12:08.650635 10211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:34395
I20260811 10:12:08.658942 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:12:08.666658 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.660780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.8.238.62:34691)
0811 10:12:08.660850 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.660852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.660953 (+ 101us) server_negotiation.cc:407] Connection header received
0811 10:12:08.661049 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.661050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.661059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.661100 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.661710 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.662176 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.662906 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.663110 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.663197 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.664334 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.664336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.664337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.664344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.665468 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.665833 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.665835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.665836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.665897 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.666107 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.666108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.666109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.666262 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.666384 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.666477 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.666558 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:12:08.669108 10211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:08.670869 10211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:08.678133 11025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.673761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34605 (local address 127.0.0.1:44138)
0811 10:12:08.674015 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.674024 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.674069 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.674695 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.674696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.674699 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:08.674760 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.674761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.676032 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.676033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.676938 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.676941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677883 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.677936 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.677939 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.677985 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.678027 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260811 10:12:08.678148 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.674212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.8.238.2:39641)
0811 10:12:08.674414 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.674417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.674427 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:08.674473 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.674475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.674511 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.674629 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:08.674724 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.675869 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677184 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677679 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677766 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.677827 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.677994 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.678038 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260811 10:12:08.678331 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.673836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.8.238.3:34795)
0811 10:12:08.674190 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.674195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.674205 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:08.674244 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.674246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.674262 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.674430 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:08.674586 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.675510 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677012 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677722 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.678091 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.678159 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.678187 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.678233 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":97,"threads_started":1}
I20260811 10:12:08.678138 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.673976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:39641 (local address 127.0.0.1:34228)
0811 10:12:08.674143 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.674155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.674212 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.674629 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.674629 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.674632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:08.674676 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.674677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.675974 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.675975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.677102 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.677104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677778 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.677860 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.677873 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.677974 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.678043 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":68,"thread_start_us":68,"threads_started":1}
I20260811 10:12:08.678274 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.673757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34795 (local address 127.0.0.1:44676)
0811 10:12:08.673926 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.673969 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.674043 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.674438 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.674441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.674444 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:08.674512 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.674514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.675618 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.675619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.676920 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.676923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677832 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.677909 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.677913 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.677957 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.678036 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1,"thread_start_us":53,"threads_started":1}
I20260811 10:12:08.678781 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.673901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.8.238.1:34605)
0811 10:12:08.674349 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.674353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.674366 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:12:08.674410 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.674413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.674429 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.674592 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:08.674840 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.675944 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677049 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.677736 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.677798 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.677854 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.678629 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.678678 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":115,"threads_started":1}
I20260811 10:12:08.679380 10659 tablet_service.cc:1467] Processing CreateTablet for tablet b14e96e3b8244a678c24ae33bd16de4f (DEFAULT_TABLE table=test-table [id=a2e1794ee5d84b289d08eb7a2fd428d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:08.679374 10891 tablet_service.cc:1467] Processing CreateTablet for tablet b14e96e3b8244a678c24ae33bd16de4f (DEFAULT_TABLE table=test-table [id=a2e1794ee5d84b289d08eb7a2fd428d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:08.679420 10434 tablet_service.cc:1467] Processing CreateTablet for tablet b14e96e3b8244a678c24ae33bd16de4f (DEFAULT_TABLE table=test-table [id=a2e1794ee5d84b289d08eb7a2fd428d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:08.679692 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e96e3b8244a678c24ae33bd16de4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:08.679718 10891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e96e3b8244a678c24ae33bd16de4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:08.679718 10434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b14e96e3b8244a678c24ae33bd16de4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:08.683385 11038 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: Bootstrap starting.
I20260811 10:12:08.683441 11035 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Bootstrap starting.
I20260811 10:12:08.683939 11036 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: Bootstrap starting.
I20260811 10:12:08.684290 11038 tablet_bootstrap.cc:654] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:08.684419 11035 tablet_bootstrap.cc:654] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:08.684880 11036 tablet_bootstrap.cc:654] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:08.685218 11035 log.cc:824] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:08.685442 11036 log.cc:824] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:08.686122 11035 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: No bootstrap required, opened a new log
I20260811 10:12:08.686210 11035 ts_tablet_manager.cc:1397] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 10:12:08.686448 11036 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: No bootstrap required, opened a new log
I20260811 10:12:08.686510 11038 log.cc:824] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:08.686815 11036 ts_tablet_manager.cc:1397] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:12:08.687527 11038 tablet_bootstrap.cc:492] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: No bootstrap required, opened a new log
I20260811 10:12:08.687605 11038 ts_tablet_manager.cc:1397] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260811 10:12:08.688872 11036 raft_consensus.cc:348] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.689051 11036 raft_consensus.cc:374] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:08.688926 11035 raft_consensus.cc:348] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.689100 11036 raft_consensus.cc:729] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ac9c888add0456db94bd876a701bd5a, State: Initialized, Role: FOLLOWER
I20260811 10:12:08.689123 11035 raft_consensus.cc:374] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:08.689173 11035 raft_consensus.cc:729] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46cd6fe251c742fdbacc146742398e85, State: Initialized, Role: FOLLOWER
I20260811 10:12:08.689235 11036 consensus_queue.cc:260] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.689292 11035 consensus_queue.cc:260] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.689582 11036 ts_tablet_manager.cc:1428] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:08.689611 10543 heartbeater.cc:503] Master 127.8.238.62:34691 was elected leader, sending a full tablet report...
I20260811 10:12:08.689620 11035 ts_tablet_manager.cc:1428] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 10:12:08.689754 10774 heartbeater.cc:503] Master 127.8.238.62:34691 was elected leader, sending a full tablet report...
I20260811 10:12:08.690164 11038 raft_consensus.cc:348] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.690366 11038 raft_consensus.cc:374] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:08.690419 11038 raft_consensus.cc:729] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26a6d05dc454f39b68803fe90b7c972, State: Initialized, Role: FOLLOWER
I20260811 10:12:08.690536 11038 consensus_queue.cc:260] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.690806 11038 ts_tablet_manager.cc:1428] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 10:12:08.690829 11004 heartbeater.cc:503] Master 127.8.238.62:34691 was elected leader, sending a full tablet report...
I20260811 10:12:08.707618 11047 raft_consensus.cc:482] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:08.707794 11047 raft_consensus.cc:504] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.708278 11047 leader_election.cc:288] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d26a6d05dc454f39b68803fe90b7c972 (127.8.238.3:34795), 7ac9c888add0456db94bd876a701bd5a (127.8.238.1:34605)
I20260811 10:12:08.712092 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.708506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:51755 (local address 127.8.238.1:34605)
0811 10:12:08.708556 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.708558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.708765 (+ 207us) server_negotiation.cc:407] Connection header received
0811 10:12:08.708795 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.708796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.708807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.708858 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:08.709063 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.709747 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711146 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.711697 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711814 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.711825 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.711981 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.712022 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:12:08.712092 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.708464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:48251 (local address 127.8.238.3:34795)
0811 10:12:08.708525 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.708527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.708536 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:08.708662 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.708664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.708672 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.708725 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:08.708932 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.709671 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711079 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.711672 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711782 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.711796 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.711981 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.712022 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:12:08.712139 10777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.708426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34795 (local address 127.8.238.2:48251)
0811 10:12:08.708493 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.708506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.708553 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.708748 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.708750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.708753 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:08.708856 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.708858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.709747 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.709747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.710989 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.710992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.711807 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.711892 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.711899 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.711960 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.712004 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:12:08.712148 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.708435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34605 (local address 127.8.238.2:51755)
0811 10:12:08.708675 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:08.708684 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:08.708754 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:08.708883 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:08.708884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:08.708886 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:08.708997 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.708998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.709822 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.709822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.711059 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:08.711061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.711825 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.711826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:08.711892 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.711900 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:12:08.711960 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 10:12:08.712002 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":5,"thread_start_us":135,"threads_started":1}
I20260811 10:12:08.712512 10465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e96e3b8244a678c24ae33bd16de4f" candidate_uuid: "46cd6fe251c742fdbacc146742398e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac9c888add0456db94bd876a701bd5a" is_pre_election: true
I20260811 10:12:08.712694 10465 raft_consensus.cc:2454] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46cd6fe251c742fdbacc146742398e85 in term 0.
I20260811 10:12:08.712666 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e96e3b8244a678c24ae33bd16de4f" candidate_uuid: "46cd6fe251c742fdbacc146742398e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26a6d05dc454f39b68803fe90b7c972" is_pre_election: true
I20260811 10:12:08.712786 10921 raft_consensus.cc:2454] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46cd6fe251c742fdbacc146742398e85 in term 0.
I20260811 10:12:08.712996 10583 leader_election.cc:302] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46cd6fe251c742fdbacc146742398e85, 7ac9c888add0456db94bd876a701bd5a; no voters:
I20260811 10:12:08.713163 11047 raft_consensus.cc:2781] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:08.713220 11047 raft_consensus.cc:482] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:08.713245 11047 raft_consensus.cc:3037] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:08.713994 11047 raft_consensus.cc:504] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.714136 11047 leader_election.cc:288] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [CANDIDATE]: Term 1 election: Requested vote from peers d26a6d05dc454f39b68803fe90b7c972 (127.8.238.3:34795), 7ac9c888add0456db94bd876a701bd5a (127.8.238.1:34605)
I20260811 10:12:08.714293 10465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e96e3b8244a678c24ae33bd16de4f" candidate_uuid: "46cd6fe251c742fdbacc146742398e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ac9c888add0456db94bd876a701bd5a"
I20260811 10:12:08.714306 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b14e96e3b8244a678c24ae33bd16de4f" candidate_uuid: "46cd6fe251c742fdbacc146742398e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26a6d05dc454f39b68803fe90b7c972"
I20260811 10:12:08.714375 10465 raft_consensus.cc:3037] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:08.714375 10921 raft_consensus.cc:3037] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:08.715258 10921 raft_consensus.cc:2454] T b14e96e3b8244a678c24ae33bd16de4f P d26a6d05dc454f39b68803fe90b7c972 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46cd6fe251c742fdbacc146742398e85 in term 1.
I20260811 10:12:08.715258 10465 raft_consensus.cc:2454] T b14e96e3b8244a678c24ae33bd16de4f P 7ac9c888add0456db94bd876a701bd5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46cd6fe251c742fdbacc146742398e85 in term 1.
I20260811 10:12:08.715508 10585 leader_election.cc:302] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46cd6fe251c742fdbacc146742398e85, d26a6d05dc454f39b68803fe90b7c972; no voters:
I20260811 10:12:08.715611 11047 raft_consensus.cc:2781] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:08.715854 11047 raft_consensus.cc:686] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [term 1 LEADER]: Becoming Leader. State: Replica: 46cd6fe251c742fdbacc146742398e85, State: Running, Role: LEADER
I20260811 10:12:08.715979 11047 consensus_queue.cc:237] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } }
I20260811 10:12:08.717013 10211 catalog_manager.cc:5495] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46cd6fe251c742fdbacc146742398e85 (127.8.238.2). New cstate: current_term: 1 leader_uuid: "46cd6fe251c742fdbacc146742398e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d26a6d05dc454f39b68803fe90b7c972" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ac9c888add0456db94bd876a701bd5a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46cd6fe251c742fdbacc146742398e85" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39641 } health_report { overall_health: HEALTHY } } }
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
W20260811 10:12:08.727875 10775 tablet.cc:2369] T b14e96e3b8244a678c24ae33bd16de4f P 46cd6fe251c742fdbacc146742398e85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:08.729322 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.721483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.8.238.1:34605)
0811 10:12:08.721572 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.721574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.721658 (+ 84us) server_negotiation.cc:407] Connection header received
0811 10:12:08.721727 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.721728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.721738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.721782 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.722382 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.722839 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.723575 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.723848 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.723922 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.726259 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.726269 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.726269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.726283 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.728052 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.728408 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.728411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.728411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.728548 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.728764 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.728765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.728766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.728949 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.729066 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.729127 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.729193 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:08.755582 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.748299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.8.238.1:34605)
0811 10:12:08.748377 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.748379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.748446 (+ 67us) server_negotiation.cc:407] Connection header received
0811 10:12:08.748543 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.748544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.748556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.748633 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.749230 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.749726 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.750466 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.750666 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.750741 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.753119 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.753120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.753120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.753128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.754335 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.754704 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.754705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.754706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.754770 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.754980 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.754981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.754981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.755085 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.755195 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.755391 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.755475 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:08.780977 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:08.773944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.8.238.1:34605)
0811 10:12:08.774016 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:08.774019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:08.774138 (+ 119us) server_negotiation.cc:407] Connection header received
0811 10:12:08.774224 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:08.774225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:08.774235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:08.774297 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:08.774847 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.775323 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.776116 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:08.776305 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:08.776376 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:08.778603 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:08.778605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:08.778605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:08.778612 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:08.779843 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.780206 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.780207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.780208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.780267 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:08.780475 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:08.780476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:08.780476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:08.780576 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:08.780680 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:08.780785 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 10:12:08.780866 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 10:12:08.781680 10402 tablet_service.cc:2908] Rejecting scan request for tablet b14e96e3b8244a678c24ae33bd16de4f: Uninitialized: clean time has not yet been initialized
I20260811 10:12:08.782657 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10328
I20260811 10:12:08.802868 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10551
I20260811 10:12:08.814682 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10780
I20260811 10:12:08.825953 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10147
2026-08-11T10:12:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2288 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[11087](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[11087](info): set up 2 sockets
Aug 11 10:12:08 dist-test-slave-7f01 krb5kdc[11087](info): commencing operation
krb5kdc: starting...
W20260811 10:12:10.202000 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:10.324992 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:33183
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:33183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:10.408044 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:10.408279 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:10.408329 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:10.409500 11110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:10.409569 11110 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:12:10.409590 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:10.409611 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:10.409643 11110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:10.409667 11110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:10.411149 11110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:33183
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:33183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:10.411434 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:10.411667 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:10.416612 11126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.416791 11110 server_base.cc:1034] running on GCE node
W20260811 10:12:10.416944 11119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:10.416983 11118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.417332 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:10.417654 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:10.418823 11110 hybrid_clock.cc:648] HybridClock initialized: now 1786443130418808 us; error 47 us; skew 500 ppm
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:10.426429 11110 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:10.426976 11110 webserver.cc:460] Webserver started at http://127.8.238.62:35939/ using document root <none> and password file <none>
I20260811 10:12:10.427210 11110 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:10.427279 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:10.427440 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:10.428575 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "c45f1f3ff760fdf16bea02b2506d45ba"
server_key_iv: "5d27101eaf5960f9636fb521dfc78814"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.428958 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "c45f1f3ff760fdf16bea02b2506d45ba"
server_key_iv: "5d27101eaf5960f9636fb521dfc78814"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.430401 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:10.431174 11138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.431375 11110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:10.431488 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "c45f1f3ff760fdf16bea02b2506d45ba"
server_key_iv: "5d27101eaf5960f9636fb521dfc78814"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.431564 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:10.440963 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:10.442706 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:10.442873 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:10.448127 11110 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:33183
I20260811 10:12:10.448190 11262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:33183 every 8 connection(s)
I20260811 10:12:10.448501 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:10.449110 11265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:10.450001 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11110
I20260811 10:12:10.450078 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:10.450255 9144 external_mini_cluster.cc:1428] Setting key ee753515dd4ad7db41c028987a476f90
I20260811 10:12:10.452382 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Bootstrap starting.
I20260811 10:12:10.453078 11265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:10.453444 11265 log.cc:824] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:10.454178 11265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: No bootstrap required, opened a new log
I20260811 10:12:10.456555 11265 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } }
I20260811 10:12:10.456718 11265 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:10.456789 11265 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebbc0369ad6249f49ff7bbae1ca78bc0, State: Initialized, Role: FOLLOWER
I20260811 10:12:10.456996 11265 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } }
I20260811 10:12:10.457100 11265 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:10.457149 11265 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:10.457199 11265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:10.457796 11265 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } }
I20260811 10:12:10.457924 11265 leader_election.cc:302] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebbc0369ad6249f49ff7bbae1ca78bc0; no voters:
I20260811 10:12:10.458106 11265 leader_election.cc:288] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:10.458194 11273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:10.458379 11273 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [term 1 LEADER]: Becoming Leader. State: Replica: ebbc0369ad6249f49ff7bbae1ca78bc0, State: Running, Role: LEADER
I20260811 10:12:10.458393 11265 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:10.458565 11273 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } }
I20260811 10:12:10.458913 11276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } } }
I20260811 10:12:10.459007 11276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:10.459270 11285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:10.459798 11285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:10.460383 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebbc0369ad6249f49ff7bbae1ca78bc0. Latest consensus state: current_term: 1 leader_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebbc0369ad6249f49ff7bbae1ca78bc0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33183 } } }
I20260811 10:12:10.460492 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:10.461447 11285 catalog_manager.cc:1347] Generated new cluster ID: 50605aa605a0468186283e647cb4e5b6
I20260811 10:12:10.461504 11285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:10.462277 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.450805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.8.238.62:33183)
0811 10:12:10.451000 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.451010 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.451044 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:12:10.451549 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.451559 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.451624 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.451798 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.452433 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.453823 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.454632 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.455281 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.455417 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.457905 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.457909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.457909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.457921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.460289 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.460757 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.460760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.460760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.460869 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.461312 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.461313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.461314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.461647 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.461793 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.461936 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.462078 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260811 10:12:10.472566 11285 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:10.473078 11285 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:10.474756 11285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Generated new TSK 0
I20260811 10:12:10.474951 11285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:10.508548 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33183
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:10.611584 11309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:10.611860 11309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:10.611889 11309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:10.613061 11309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:10.613128 11309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:10.613163 11309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:10.613219 11309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:10.614764 11309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:10.615042 11309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:10.615278 11309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:10.620450 11320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.620532 11309 server_base.cc:1034] running on GCE node
W20260811 10:12:10.620586 11324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:10.620467 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.620891 11309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:10.621158 11309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:10.622336 11309 hybrid_clock.cc:648] HybridClock initialized: now 1786443130622321 us; error 30 us; skew 500 ppm
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:10.625075 11309 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:10.625550 11309 webserver.cc:460] Webserver started at http://127.8.238.1:34159/ using document root <none> and password file <none>
I20260811 10:12:10.625797 11309 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:10.625865 11309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:10.626014 11309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:10.627070 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "4169d06948ac41cf83159c606289adbf"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "6fbbe053e61d57f4f4eb02bfab19f9fa"
server_key_iv: "368d15b2991b2ee12b149bd858fa6fb9"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.627436 11309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "4169d06948ac41cf83159c606289adbf"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "6fbbe053e61d57f4f4eb02bfab19f9fa"
server_key_iv: "368d15b2991b2ee12b149bd858fa6fb9"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.628844 11309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:10.629621 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.629844 11309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:10.629956 11309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "4169d06948ac41cf83159c606289adbf"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "6fbbe053e61d57f4f4eb02bfab19f9fa"
server_key_iv: "368d15b2991b2ee12b149bd858fa6fb9"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.630031 11309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:10.640944 11309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:10.642940 11309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:10.643113 11309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:10.643616 11309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:10.643677 11309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.643711 11309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:10.643834 11309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.651494 11309 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:36637
I20260811 10:12:10.651558 11548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:36637 every 8 connection(s)
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:10.651957 11309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:10.654666 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11309
I20260811 10:12:10.654774 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:10.654980 9144 external_mini_cluster.cc:1428] Setting key 4591ca79cc377ddedec128958133d3d0
I20260811 10:12:10.655503 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.644390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:59787 (local address 127.8.238.62:33183)
0811 10:12:10.644456 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.644459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.644700 (+ 241us) server_negotiation.cc:407] Connection header received
0811 10:12:10.645228 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.645230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.645244 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.645334 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.646331 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.647262 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.649110 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.649451 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.649532 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.652562 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.652564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.652564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.652571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.654012 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.654371 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.654372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.654372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.654425 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.654762 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.654763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.654764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.654913 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.654967 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.655313 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.655410 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:12:10.655682 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.644314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.1:59787)
0811 10:12:10.644575 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.644605 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.645120 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.645407 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.645411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.645886 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.646223 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.646230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.647380 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.647382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.648998 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.649002 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.649618 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.649619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.649779 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.650413 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:10.650435 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:10.652473 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:10.654084 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.654091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.654092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.654305 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.654524 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.654525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.654526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.654669 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.654981 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:10.654988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:10.655150 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 10:12:10.655292 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 10:12:10.655446 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
I20260811 10:12:10.656200 11550 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33183
I20260811 10:12:10.656353 11550 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:10.656608 11550 heartbeater.cc:511] Master 127.8.238.62:33183 requested a full tablet report, sending...
I20260811 10:12:10.657171 11173 ts_manager.cc:194] Registered new tserver with Master: 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637)
I20260811 10:12:10.658087 11173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:59787
I20260811 10:12:10.662843 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.657390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:42223 (local address 127.8.238.62:33183)
0811 10:12:10.657429 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.657431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.657440 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:10.657505 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.657506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.657516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.657570 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.657921 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.658477 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.659174 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.659431 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.659520 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.660391 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.660393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.660393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.660401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.661690 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.662054 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.662055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.662056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.662110 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.662327 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.662328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.662328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.662439 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.662509 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.662684 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.662755 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:10.662868 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.657254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.1:42223)
0811 10:12:10.657381 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.657391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.657445 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.657575 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.657576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.657754 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.657833 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.657834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.658548 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.658549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.659088 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.659090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.659524 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.659525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.659594 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.660015 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:10.660020 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:10.660304 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:10.661775 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.661777 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.661777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.661965 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.662187 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.662188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.662188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.662247 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.662514 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:10.662515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:10.662572 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 10:12:10.662655 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:12:10.662756 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:10.682785 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33183
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:10.767385 11570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:10.767580 11570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:10.767604 11570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:10.768774 11570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:10.768815 11570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:10.768838 11570 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:10.768898 11570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:10.770308 11570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:10.770558 11570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:10.770789 11570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:10.775913 11575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:10.775952 11581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.775955 11570 server_base.cc:1034] running on GCE node
W20260811 10:12:10.775910 11576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.776342 11570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:10.776548 11570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:10.777688 11570 hybrid_clock.cc:648] HybridClock initialized: now 1786443130777673 us; error 30 us; skew 500 ppm
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:10.780383 11570 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:10.780818 11570 webserver.cc:460] Webserver started at http://127.8.238.2:35085/ using document root <none> and password file <none>
I20260811 10:12:10.781059 11570 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:10.781132 11570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:10.781296 11570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:10.782356 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "34a4870179681ca5add30273df0421b8"
server_key_iv: "a7bef340d1881d84ad2f190efbcd7fb5"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.782728 11570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "34a4870179681ca5add30273df0421b8"
server_key_iv: "a7bef340d1881d84ad2f190efbcd7fb5"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.784137 11570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:10.784878 11599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.785053 11570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:10.785167 11570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "34a4870179681ca5add30273df0421b8"
server_key_iv: "a7bef340d1881d84ad2f190efbcd7fb5"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.785243 11570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:10.810668 11570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:10.812767 11570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:10.812958 11570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:10.813580 11570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:10.813650 11570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.813694 11570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:10.813726 11570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:10.821259 11570 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:41485
I20260811 10:12:10.821341 11807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:41485 every 8 connection(s)
I20260811 10:12:10.821722 11570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:10.824783 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.814102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:36173 (local address 127.8.238.62:33183)
0811 10:12:10.814173 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.814176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.814398 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:12:10.814896 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.814898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.814914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.815004 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.815854 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.816726 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.818371 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.818746 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.818854 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.821695 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.821697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.821697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.821706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.823279 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.823779 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.823780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.823781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.823856 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.824123 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.824124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.824125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.824249 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.824324 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.824606 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.824690 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:12:10.824959 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.814077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.2:36173)
0811 10:12:10.814263 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.814302 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.814790 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.815091 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.815095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.815442 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.815728 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.815735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.816841 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.816843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.818271 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.818274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.818872 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.818873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.819021 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.819592 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:10.819601 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:10.821598 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:10.823390 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.823398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.823400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.823671 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.823936 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.823937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.823937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.824050 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.824336 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:10.824344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:10.824476 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 10:12:10.824580 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 10:12:10.824735 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260811 10:12:10.825425 11812 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33183
I20260811 10:12:10.825506 11812 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:10.825686 11812 heartbeater.cc:511] Master 127.8.238.62:33183 requested a full tablet report, sending...
I20260811 10:12:10.826079 11171 ts_manager.cc:194] Registered new tserver with Master: b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485)
I20260811 10:12:10.826594 11171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:36173
I20260811 10:12:10.828156 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11570
I20260811 10:12:10.828253 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:10.828437 9144 external_mini_cluster.cc:1428] Setting key 1e8ead2b5342368f87f92859f52e0b92
I20260811 10:12:10.832998 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.826437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.2:44747)
0811 10:12:10.826577 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.826588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.826653 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.826815 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.826816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.827039 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.827098 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.827099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.827876 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.827876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.828660 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.828662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.829009 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.829010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.829085 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.829461 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:10.829466 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:10.829899 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:10.831823 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.831824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.831825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.832001 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.832208 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.832209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.832209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.832264 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.832625 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:10.832626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:10.832726 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 10:12:10.832796 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:12:10.832927 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260811 10:12:10.833079 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.826534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:44747 (local address 127.8.238.62:33183)
0811 10:12:10.826574 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.826576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.826643 (+ 67us) server_negotiation.cc:407] Connection header received
0811 10:12:10.826726 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.826727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.826737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.826797 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.827173 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.827788 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.828719 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.828939 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.828991 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.830007 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.830008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.830008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.830014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.831705 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.832074 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.832075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.832076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.832140 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.832325 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.832326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.832327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.832548 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.832626 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.832861 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.832948 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:10.854529 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33183
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:38591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:10.938186 11832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:10.938381 11832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:10.938405 11832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:10.939499 11832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:10.939534 11832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:10.939554 11832 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:10.939572 11832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:10.941010 11832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:38591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:33183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:10.941195 11832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:10.941368 11832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:10.946337 11842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.946573 11832 server_base.cc:1034] running on GCE node
W20260811 10:12:10.946307 11839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:10.946312 11840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:10.946885 11832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:10.947145 11832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:10.948305 11832 hybrid_clock.cc:648] HybridClock initialized: now 1786443130948301 us; error 38 us; skew 500 ppm
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:10.950888 11832 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:10.951345 11832 webserver.cc:460] Webserver started at http://127.8.238.3:41423/ using document root <none> and password file <none>
I20260811 10:12:10.951541 11832 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:10.951583 11832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:10.951673 11832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:10.952730 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "8d7c3b5d18a94541a8956f3ff4edb45f"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "b967dba13b05430031f41a58030c9199"
server_key_iv: "a7cc0bfa2caa8a768d725f51ea1c3660"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.953033 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "8d7c3b5d18a94541a8956f3ff4edb45f"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "b967dba13b05430031f41a58030c9199"
server_key_iv: "a7cc0bfa2caa8a768d725f51ea1c3660"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.954308 11832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:10.954993 11850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.955179 11832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:10.955271 11832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "8d7c3b5d18a94541a8956f3ff4edb45f"
format_stamp: "Formatted at 2026-08-11 10:12:10 on dist-test-slave-7f01"
server_key: "b967dba13b05430031f41a58030c9199"
server_key_iv: "a7cc0bfa2caa8a768d725f51ea1c3660"
server_key_version: "encryptionkey@0"
I20260811 10:12:10.955327 11832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:10.982604 11832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:10.985054 11832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:10.985176 11832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:10.985678 11832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:10.985728 11832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.985755 11832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:10.985769 11832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:10.993289 11832 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:36227
I20260811 10:12:10.993464 12058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:36227 every 8 connection(s)
I20260811 10:12:10.993762 11832 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:10 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:10.997078 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.986222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:48505 (local address 127.8.238.62:33183)
0811 10:12:10.986299 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.986301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.986526 (+ 225us) server_negotiation.cc:407] Connection header received
0811 10:12:10.987040 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.987042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.987058 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.987140 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.988012 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.988898 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.990481 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.990801 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.990874 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.994477 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:10.994478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:10.994479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:10.994486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:10.995629 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.996063 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.996065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.996066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.996130 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:10.996403 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:10.996404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:10.996404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:10.996530 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:10.996611 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:10.996887 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 10:12:10.996989 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:12:10.997238 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.986188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.3:48505)
0811 10:12:10.986381 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.986422 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.986938 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.987234 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.987239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.987596 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.987909 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.987917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.989014 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.989015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.990371 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.990376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:10.990989 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:10.990991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:10.991148 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:10.991854 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:10.991864 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:10.994365 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:10.995713 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.995721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.995723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.995981 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.996209 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:10.996210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:10.996210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:10.996326 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:10.996599 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:10.996606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:10.996749 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 10:12:10.996851 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 10:12:10.997009 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260811 10:12:10.997803 12059 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33183
I20260811 10:12:10.997888 12059 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:10.998091 12059 heartbeater.cc:511] Master 127.8.238.62:33183 requested a full tablet report, sending...
I20260811 10:12:10.998468 11171 ts_manager.cc:194] Registered new tserver with Master: 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3:36227)
I20260811 10:12:10.998976 11171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:48505
I20260811 10:12:10.999536 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11832
I20260811 10:12:10.999640 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:10.999840 9144 external_mini_cluster.cc:1428] Setting key 934df18b112f692a1bde30722926bbb3
I20260811 10:12:11.000751 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:12:11.005229 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.998942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:35813 (local address 127.8.238.62:33183)
0811 10:12:10.999002 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:10.999003 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:10.999121 (+ 118us) server_negotiation.cc:407] Connection header received
0811 10:12:10.999185 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:10.999186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:10.999195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:10.999240 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:10.999627 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.000299 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.001149 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.001350 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.001434 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.002363 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:11.002364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:11.002365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:11.002371 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:11.004077 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.004413 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.004415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.004415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.004478 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.004699 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.004699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.004700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.004802 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:11.004852 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.005100 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.005168 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:12:11.005282 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:10.998882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.8.238.3:35813)
0811 10:12:10.999063 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:10.999074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:10.999137 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:10.999253 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:10.999254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:10.999468 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:10.999531 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:10.999533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.000412 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.000413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.001059 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.001061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.001437 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.001438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.001519 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.001966 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:11.001971 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:11.002244 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:11.004163 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:11.004164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:11.004165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:11.004339 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:11.004556 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:11.004557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:11.004558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:11.004631 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:11.004867 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:11.004869 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:11.004922 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.005005 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.005126 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[11087](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:11.026705 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.019871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.8.238.62:33183)
0811 10:12:11.019930 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.019932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.019942 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:11.019972 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.019973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.019987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.020038 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:11.020630 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.021127 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.021846 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.022030 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.022083 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.024301 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:11.024302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:11.024303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:11.024317 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:11.025650 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.025963 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.025964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.025964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.026034 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.026234 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.026234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.026235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.026347 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:11.026471 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.026555 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.026615 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:12:11.028458 11171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:11.030174 11171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:11.037328 12092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.032630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36227 (local address 127.0.0.1:57170)
0811 10:12:11.033000 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.033037 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.033104 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.033472 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.033475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.033477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.033547 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.033548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.034919 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.034920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.036069 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.036072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.037013 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.037015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.037113 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037118 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.037150 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.037218 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":86,"thread_start_us":54,"threads_started":1}
I20260811 10:12:11.037432 12101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.032631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36637 (local address 127.0.0.1:40378)
0811 10:12:11.032997 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.033030 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.033111 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.033562 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.033564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.033566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.033613 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.033614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.035006 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.035007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.036250 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.036254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.037217 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.037218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.037282 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037286 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.037334 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.037371 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":26,"thread_start_us":82,"threads_started":1}
I20260811 10:12:11.037366 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.033103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.8.238.2:41485)
0811 10:12:11.033275 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.033280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.033445 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:12:11.033580 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.033583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.033595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.033682 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.033840 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.034669 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.036391 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.036879 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.036973 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037034 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.037210 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.037251 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260811 10:12:11.037623 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.032849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:41485 (local address 127.0.0.1:57520)
0811 10:12:11.033338 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.033355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.033472 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.033681 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.033681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.033683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.033753 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.033755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.034762 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.034763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.036293 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.036295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.036973 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.036975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.037067 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037082 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.037188 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.037236 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":36,"thread_start_us":75,"threads_started":1}
I20260811 10:12:11.037714 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.032828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.8.238.3:36227)
0811 10:12:11.033255 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.033260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.033269 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:11.033320 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.033322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.033339 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.033630 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.033695 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.034832 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.036160 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.036902 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.037287 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037353 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.037378 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.037415 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":98,"threads_started":1}
I20260811 10:12:11.037793 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.032738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.8.238.1:36637)
0811 10:12:11.032899 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.032903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.033107 (+ 204us) server_negotiation.cc:407] Connection header received
0811 10:12:11.033384 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.033387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.033405 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.033477 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.033690 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.034831 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.036348 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.037063 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.037123 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.037183 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.037635 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.037682 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260811 10:12:11.038363 11948 tablet_service.cc:1467] Processing CreateTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 (DEFAULT_TABLE table=test-table [id=c91c82397985413bac8cadeb0dc9aa18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.038452 11429 tablet_service.cc:1467] Processing CreateTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 (DEFAULT_TABLE table=test-table [id=c91c82397985413bac8cadeb0dc9aa18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.038749 11948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e324e9a0d3cf48f18ee119da4abd2a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.038885 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e324e9a0d3cf48f18ee119da4abd2a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.038838 11681 tablet_service.cc:1467] Processing CreateTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 (DEFAULT_TABLE table=test-table [id=c91c82397985413bac8cadeb0dc9aa18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.039153 11681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e324e9a0d3cf48f18ee119da4abd2a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.042169 12108 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Bootstrap starting.
I20260811 10:12:11.042497 12109 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Bootstrap starting.
I20260811 10:12:11.042894 12108 tablet_bootstrap.cc:654] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.043033 12110 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Bootstrap starting.
I20260811 10:12:11.043385 12108 log.cc:824] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:11.043566 12109 tablet_bootstrap.cc:654] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.043720 12110 tablet_bootstrap.cc:654] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.044026 12109 log.cc:824] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:11.044216 12110 log.cc:824] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:11.044426 12108 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: No bootstrap required, opened a new log
I20260811 10:12:11.044516 12108 ts_tablet_manager.cc:1397] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:11.044832 12109 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: No bootstrap required, opened a new log
I20260811 10:12:11.044919 12109 ts_tablet_manager.cc:1397] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 10:12:11.045055 12110 tablet_bootstrap.cc:492] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: No bootstrap required, opened a new log
I20260811 10:12:11.045123 12110 ts_tablet_manager.cc:1397] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:12:11.046283 12109 raft_consensus.cc:348] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.046406 12109 raft_consensus.cc:374] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.046437 12109 raft_consensus.cc:729] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7c3b5d18a94541a8956f3ff4edb45f, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.046557 12109 consensus_queue.cc:260] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.046645 12108 raft_consensus.cc:348] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.046850 12109 ts_tablet_manager.cc:1428] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:12:11.046847 12108 raft_consensus.cc:374] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.046912 12108 raft_consensus.cc:729] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4169d06948ac41cf83159c606289adbf, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.047021 12108 consensus_queue.cc:260] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.047327 12108 ts_tablet_manager.cc:1428] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:11.047350 11550 heartbeater.cc:503] Master 127.8.238.62:33183 was elected leader, sending a full tablet report...
I20260811 10:12:11.047447 12059 heartbeater.cc:503] Master 127.8.238.62:33183 was elected leader, sending a full tablet report...
I20260811 10:12:11.047482 12110 raft_consensus.cc:348] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.047618 12110 raft_consensus.cc:374] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.047652 12110 raft_consensus.cc:729] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d4aaebecd64ce5877a368facd1e7c9, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.047887 12110 consensus_queue.cc:260] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.048239 11812 heartbeater.cc:503] Master 127.8.238.62:33183 was elected leader, sending a full tablet report...
I20260811 10:12:11.048506 12110 ts_tablet_manager.cc:1428] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260811 10:12:11.072551 11814 tablet.cc:2369] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:11.076112 12120 raft_consensus.cc:482] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:11.076206 12120 raft_consensus.cc:504] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.076587 12120 leader_election.cc:288] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637), b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485)
I20260811 10:12:11.079694 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.076902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:57621 (local address 127.8.238.1:36637)
0811 10:12:11.076952 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.076954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.076963 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:11.077031 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.077032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.077044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.077099 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.077269 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.077958 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.078899 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.079265 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.079349 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.079360 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.079588 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.079625 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:11.079694 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.076793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36637 (local address 127.8.238.3:57621)
0811 10:12:11.076874 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.076889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.076963 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.077113 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.077114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.077116 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.077194 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.077195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.078038 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.078039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.078791 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.078793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.079350 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.079351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.079502 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.079506 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.079580 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.079625 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 10:12:11.079819 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.077017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:51581 (local address 127.8.238.2:41485)
0811 10:12:11.077074 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.077076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.077195 (+ 119us) server_negotiation.cc:407] Connection header received
0811 10:12:11.077258 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.077259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.077271 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.077351 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.077525 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.078129 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.079070 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.079442 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.079526 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.079535 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.079696 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.079757 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:12:11.079833 12125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.076941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:41485 (local address 127.8.238.3:51581)
0811 10:12:11.077110 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.077119 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.077198 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.077362 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.077362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.077364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.077440 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.077442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.078216 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.078217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.078971 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.078974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.079530 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.079531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.079618 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.079621 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.079689 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.079727 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260811 10:12:11.080224 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e324e9a0d3cf48f18ee119da4abd2a17" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" is_pre_election: true
I20260811 10:12:11.080238 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e324e9a0d3cf48f18ee119da4abd2a17" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4169d06948ac41cf83159c606289adbf" is_pre_election: true
I20260811 10:12:11.080379 11466 raft_consensus.cc:2454] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 0.
I20260811 10:12:11.080377 11718 raft_consensus.cc:2454] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 0.
I20260811 10:12:11.080614 11852 leader_election.cc:302] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4169d06948ac41cf83159c606289adbf, 8d7c3b5d18a94541a8956f3ff4edb45f; no voters:
I20260811 10:12:11.080740 12120 raft_consensus.cc:2781] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:11.080798 12120 raft_consensus.cc:482] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:11.080816 12120 raft_consensus.cc:3037] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.081664 12120 raft_consensus.cc:504] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.081800 12120 leader_election.cc:288] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 election: Requested vote from peers 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637), b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485)
I20260811 10:12:11.081991 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e324e9a0d3cf48f18ee119da4abd2a17" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
I20260811 10:12:11.081979 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e324e9a0d3cf48f18ee119da4abd2a17" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4169d06948ac41cf83159c606289adbf"
I20260811 10:12:11.082060 11718 raft_consensus.cc:3037] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.082063 11466 raft_consensus.cc:3037] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.082955 11718 raft_consensus.cc:2454] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 1.
I20260811 10:12:11.082959 11466 raft_consensus.cc:2454] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 1.
I20260811 10:12:11.083186 11852 leader_election.cc:302] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d7c3b5d18a94541a8956f3ff4edb45f, b5d4aaebecd64ce5877a368facd1e7c9; no voters:
I20260811 10:12:11.083319 12120 raft_consensus.cc:2781] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:11.083480 12120 raft_consensus.cc:686] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 LEADER]: Becoming Leader. State: Replica: 8d7c3b5d18a94541a8956f3ff4edb45f, State: Running, Role: LEADER
I20260811 10:12:11.083555 12120 consensus_queue.cc:237] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } }
I20260811 10:12:11.084434 11171 catalog_manager.cc:5495] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3). New cstate: current_term: 1 leader_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:11.096141 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.092982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.8.238.3:36227)
0811 10:12:11.093045 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.093048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.093107 (+ 59us) server_negotiation.cc:407] Connection header received
0811 10:12:11.093207 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.093208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.093230 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.093292 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:11.093483 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.094007 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.094894 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.095124 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.095218 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.095462 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.095570 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.095667 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.095716 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.095765 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:11.098114 11466 raft_consensus.cc:1261] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Refusing update from remote peer 8d7c3b5d18a94541a8956f3ff4edb45f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.098114 11718 raft_consensus.cc:1261] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Refusing update from remote peer 8d7c3b5d18a94541a8956f3ff4edb45f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.098477 12128 consensus_queue.cc:1035] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.098627 12120 consensus_queue.cc:1035] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.103488 12135 mvcc.cc:205] Tried to move back new op lower bound from 7317271064974897152 to 7317271064919093248. Current Snapshot: MvccSnapshot[applied={T|T < 7317271064974897152}]
I20260811 10:12:11.107234 11171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:11.107378 11171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:11.108345 11171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Sending DeleteTablet for 3 replicas of tablet e324e9a0d3cf48f18ee119da4abd2a17
I20260811 10:12:11.108872 11948 tablet_service.cc:1514] Processing DeleteTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57170
I20260811 10:12:11.108891 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:40378
I20260811 10:12:11.108989 11171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:11.109141 11171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:11.109120 11681 tablet_service.cc:1514] Processing DeleteTablet for tablet e324e9a0d3cf48f18ee119da4abd2a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57520
I20260811 10:12:11.109434 12151 tablet_replica.cc:330] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: stopping tablet replica
I20260811 10:12:11.109579 12151 raft_consensus.cc:2229] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:11.109752 12151 raft_consensus.cc:2258] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.109973 12156 tablet_replica.cc:330] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: stopping tablet replica
I20260811 10:12:11.110074 12150 tablet_replica.cc:330] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: stopping tablet replica
I20260811 10:12:11.110143 12156 raft_consensus.cc:2229] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:11.110244 12150 raft_consensus.cc:2229] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:11.110349 12156 raft_consensus.cc:2258] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.110432 12150 raft_consensus.cc:2258] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.110625 12151 ts_tablet_manager.cc:1905] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.110831 11948 tablet_service.cc:1467] Processing CreateTablet for tablet 03e83ba448c1440a83e5ce67b884dbde (DEFAULT_TABLE table=test-table [id=7b508b8bd2cf48d49085db4b71063a17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.110929 12156 ts_tablet_manager.cc:1905] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.110970 11948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e83ba448c1440a83e5ce67b884dbde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.111121 11681 tablet_service.cc:1467] Processing CreateTablet for tablet 03e83ba448c1440a83e5ce67b884dbde (DEFAULT_TABLE table=test-table [id=7b508b8bd2cf48d49085db4b71063a17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.111213 11681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e83ba448c1440a83e5ce67b884dbde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.112680 12110 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Bootstrap starting.
I20260811 10:12:11.112774 12109 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Bootstrap starting.
I20260811 10:12:11.110947 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 03e83ba448c1440a83e5ce67b884dbde (DEFAULT_TABLE table=test-table [id=7b508b8bd2cf48d49085db4b71063a17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:11.113216 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e83ba448c1440a83e5ce67b884dbde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.113489 12110 tablet_bootstrap.cc:654] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.113657 12109 tablet_bootstrap.cc:654] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.113986 12156 ts_tablet_manager.cc:1918] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:11.114056 12156 log.cc:1196] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/e324e9a0d3cf48f18ee119da4abd2a17
I20260811 10:12:11.114107 12110 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: No bootstrap required, opened a new log
I20260811 10:12:11.114147 12110 ts_tablet_manager.cc:1397] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:11.114354 12156 ts_tablet_manager.cc:1939] T e324e9a0d3cf48f18ee119da4abd2a17 P b5d4aaebecd64ce5877a368facd1e7c9: Deleting consensus metadata
I20260811 10:12:11.114346 12110 raft_consensus.cc:348] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.114454 12110 raft_consensus.cc:374] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.114481 12110 raft_consensus.cc:729] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d4aaebecd64ce5877a368facd1e7c9, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.114532 12110 consensus_queue.cc:260] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.114630 12110 ts_tablet_manager.cc:1428] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:11.114959 11139 catalog_manager.cc:4841] TS b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485): tablet e324e9a0d3cf48f18ee119da4abd2a17 (table test-table [id=c91c82397985413bac8cadeb0dc9aa18]) successfully deleted
I20260811 10:12:11.115669 12108 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Bootstrap starting.
I20260811 10:12:11.116308 12108 tablet_bootstrap.cc:654] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.116509 12150 ts_tablet_manager.cc:1905] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.116989 12108 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: No bootstrap required, opened a new log
I20260811 10:12:11.117058 12108 ts_tablet_manager.cc:1397] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:11.117210 12108 raft_consensus.cc:348] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.117298 12108 raft_consensus.cc:374] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.117324 12108 raft_consensus.cc:729] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4169d06948ac41cf83159c606289adbf, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.117367 12108 consensus_queue.cc:260] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.117466 12108 ts_tablet_manager.cc:1428] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:11.118512 12150 ts_tablet_manager.cc:1918] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:11.118594 12150 log.cc:1196] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/e324e9a0d3cf48f18ee119da4abd2a17
I20260811 10:12:11.118891 12150 ts_tablet_manager.cc:1939] T e324e9a0d3cf48f18ee119da4abd2a17 P 4169d06948ac41cf83159c606289adbf: Deleting consensus metadata
I20260811 10:12:11.119387 11139 catalog_manager.cc:4841] TS 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637): tablet e324e9a0d3cf48f18ee119da4abd2a17 (table test-table [id=c91c82397985413bac8cadeb0dc9aa18]) successfully deleted
I20260811 10:12:11.120265 12151 ts_tablet_manager.cc:1918] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:11.120407 12151 log.cc:1196] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/e324e9a0d3cf48f18ee119da4abd2a17
I20260811 10:12:11.120419 12109 tablet_bootstrap.cc:492] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: No bootstrap required, opened a new log
I20260811 10:12:11.120500 12109 ts_tablet_manager.cc:1397] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260811 10:12:11.120702 12151 ts_tablet_manager.cc:1939] T e324e9a0d3cf48f18ee119da4abd2a17 P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting consensus metadata
I20260811 10:12:11.120687 12109 raft_consensus.cc:348] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.120798 12109 raft_consensus.cc:374] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.120831 12109 raft_consensus.cc:729] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7c3b5d18a94541a8956f3ff4edb45f, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.120879 12109 consensus_queue.cc:260] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.120970 12109 ts_tablet_manager.cc:1428] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:11.121120 11141 catalog_manager.cc:4841] TS 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3:36227): tablet e324e9a0d3cf48f18ee119da4abd2a17 (table test-table [id=c91c82397985413bac8cadeb0dc9aa18]) successfully deleted
W20260811 10:12:11.152714 11551 tablet.cc:2369] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:11.218567 12145 raft_consensus.cc:482] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:11.218700 12145 raft_consensus.cc:504] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.218963 12145 leader_election.cc:288] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637), b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485)
I20260811 10:12:11.219202 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e83ba448c1440a83e5ce67b884dbde" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4169d06948ac41cf83159c606289adbf" is_pre_election: true
I20260811 10:12:11.219231 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e83ba448c1440a83e5ce67b884dbde" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" is_pre_election: true
I20260811 10:12:11.219331 11466 raft_consensus.cc:2454] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 0.
I20260811 10:12:11.219362 11718 raft_consensus.cc:2454] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 0.
I20260811 10:12:11.219528 11852 leader_election.cc:302] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d7c3b5d18a94541a8956f3ff4edb45f, b5d4aaebecd64ce5877a368facd1e7c9; no voters:
I20260811 10:12:11.219650 12145 raft_consensus.cc:2781] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:11.219697 12145 raft_consensus.cc:482] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:11.219719 12145 raft_consensus.cc:3037] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.220530 12145 raft_consensus.cc:504] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.220672 12145 leader_election.cc:288] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 election: Requested vote from peers 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637), b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485)
I20260811 10:12:11.220844 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e83ba448c1440a83e5ce67b884dbde" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4169d06948ac41cf83159c606289adbf"
I20260811 10:12:11.220873 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e83ba448c1440a83e5ce67b884dbde" candidate_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
I20260811 10:12:11.220942 11718 raft_consensus.cc:3037] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.220935 11466 raft_consensus.cc:3037] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.221758 11718 raft_consensus.cc:2454] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 1.
I20260811 10:12:11.221760 11466 raft_consensus.cc:2454] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d7c3b5d18a94541a8956f3ff4edb45f in term 1.
I20260811 10:12:11.221936 11852 leader_election.cc:302] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d7c3b5d18a94541a8956f3ff4edb45f, b5d4aaebecd64ce5877a368facd1e7c9; no voters:
I20260811 10:12:11.222020 12145 raft_consensus.cc:2781] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:11.222083 12145 raft_consensus.cc:686] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 LEADER]: Becoming Leader. State: Replica: 8d7c3b5d18a94541a8956f3ff4edb45f, State: Running, Role: LEADER
I20260811 10:12:11.222136 12145 consensus_queue.cc:237] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.222837 11171 catalog_manager.cc:5495] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3). New cstate: current_term: 1 leader_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } health_report { overall_health: HEALTHY } } }
W20260811 10:12:11.244415 12060 tablet.cc:2369] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443130, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:11.255374 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.248501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33183 (local address 127.0.0.1:46678)
0811 10:12:11.248692 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.248703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.248763 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.248910 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.248910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.249246 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:11.249299 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.249302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.249942 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.249943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.250549 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.250550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.250882 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.250883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.250925 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.251503 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:11.251508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:11.253011 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:11.254399 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:11.254407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:11.254409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:11.254618 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:11.254801 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:11.254801 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:11.254802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:11.254867 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:11.255078 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:11.255085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:11.255125 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.255177 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.255267 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260811 10:12:11.255442 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.248597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.8.238.62:33183)
0811 10:12:11.248668 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.248670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.248764 (+ 94us) server_negotiation.cc:407] Connection header received
0811 10:12:11.248833 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.248834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.248843 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.248889 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:11.249379 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.249870 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.250629 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.250812 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.250865 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.253100 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:11.253101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:11.253101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:11.253108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:11.254320 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.254690 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.254690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.254691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.254742 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.254922 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.254923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.254923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.255022 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:11.255072 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.255211 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.255313 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":39}
I20260811 10:12:11.257644 11171 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:46678:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:12:11.257941 11171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:11.259604 11948 tablet_service.cc:1467] Processing CreateTablet for tablet 929d648f40ab4836ae65972de69a5bed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91328966d22f4d02b86a41bce6fa2f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:11.259622 11429 tablet_service.cc:1467] Processing CreateTablet for tablet 929d648f40ab4836ae65972de69a5bed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91328966d22f4d02b86a41bce6fa2f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:11.259641 11681 tablet_service.cc:1467] Processing CreateTablet for tablet 929d648f40ab4836ae65972de69a5bed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91328966d22f4d02b86a41bce6fa2f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:11.259825 11681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929d648f40ab4836ae65972de69a5bed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.259833 11429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929d648f40ab4836ae65972de69a5bed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.259863 11948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 929d648f40ab4836ae65972de69a5bed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:11.261505 12109 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: Bootstrap starting.
I20260811 10:12:11.261596 12108 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: Bootstrap starting.
I20260811 10:12:11.261598 12110 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: Bootstrap starting.
I20260811 10:12:11.261879 12109 tablet_bootstrap.cc:654] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.262126 12108 tablet_bootstrap.cc:654] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.262197 12110 tablet_bootstrap.cc:654] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:11.263156 12108 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: No bootstrap required, opened a new log
I20260811 10:12:11.263228 12108 ts_tablet_manager.cc:1397] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:11.263497 12110 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: No bootstrap required, opened a new log
I20260811 10:12:11.263521 12109 tablet_bootstrap.cc:492] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: No bootstrap required, opened a new log
I20260811 10:12:11.263571 12110 ts_tablet_manager.cc:1397] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:12:11.263590 12109 ts_tablet_manager.cc:1397] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:12:11.263666 12108 raft_consensus.cc:348] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.263767 12108 raft_consensus.cc:374] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.263790 12108 raft_consensus.cc:729] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4169d06948ac41cf83159c606289adbf, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.263762 12109 raft_consensus.cc:348] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.263829 12109 raft_consensus.cc:374] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.263859 12109 raft_consensus.cc:729] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7c3b5d18a94541a8956f3ff4edb45f, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.263846 12108 consensus_queue.cc:260] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.263899 12109 consensus_queue.cc:260] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.263950 12108 ts_tablet_manager.cc:1428] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:11.263911 12110 raft_consensus.cc:348] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.263983 12109 ts_tablet_manager.cc:1428] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:11.263989 12110 raft_consensus.cc:374] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:11.264011 12110 raft_consensus.cc:729] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d4aaebecd64ce5877a368facd1e7c9, State: Initialized, Role: FOLLOWER
I20260811 10:12:11.264051 12110 consensus_queue.cc:260] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.264139 12110 ts_tablet_manager.cc:1428] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:11.264102 12122 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.264176 12122 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.264310 12145 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.264256 12123 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.264415 12145 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.264424 12123 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.288300 12122 raft_consensus.cc:482] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:11.288381 12122 raft_consensus.cc:504] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.288834 12122 leader_election.cc:288] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485), 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3:36227)
I20260811 10:12:11.292994 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.289080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:47789 (local address 127.8.238.3:36227)
0811 10:12:11.289128 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.289130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.289198 (+ 68us) server_negotiation.cc:407] Connection header received
0811 10:12:11.289331 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.289333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.289344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.289396 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.289628 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.290362 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292022 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.292572 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292647 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.292658 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.292905 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.292942 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:12:11.293087 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.288978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36227 (local address 127.8.238.1:47789)
0811 10:12:11.289120 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.289132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.289227 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.289406 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.289408 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.289410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.289530 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.289532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.290447 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.290448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.291944 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.291945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.292682 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.292801 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.292805 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.292898 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.292975 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":62,"thread_start_us":73,"threads_started":1}
I20260811 10:12:11.293179 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.289026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:35355 (local address 127.8.238.2:41485)
0811 10:12:11.289124 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.289128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.289346 (+ 218us) server_negotiation.cc:407] Connection header received
0811 10:12:11.289437 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.289438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.289447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.289477 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:11.289832 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.290514 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292022 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.292572 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292647 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.292659 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.293061 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.293114 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 10:12:11.293370 12180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.288967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:41485 (local address 127.8.238.1:35355)
0811 10:12:11.289120 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.289132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.289345 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.289573 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.289574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.289576 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:11.289747 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.289749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.290590 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.290591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.291929 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.291933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.292789 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.292790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.292977 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.292981 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.293039 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.293071 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":46,"thread_start_us":78,"threads_started":1}
I20260811 10:12:11.293520 11983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929d648f40ab4836ae65972de69a5bed" candidate_uuid: "4169d06948ac41cf83159c606289adbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" is_pre_election: true
I20260811 10:12:11.293660 11983 raft_consensus.cc:2454] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4169d06948ac41cf83159c606289adbf in term 0.
I20260811 10:12:11.293795 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929d648f40ab4836ae65972de69a5bed" candidate_uuid: "4169d06948ac41cf83159c606289adbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" is_pre_election: true
I20260811 10:12:11.293926 11718 raft_consensus.cc:2454] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4169d06948ac41cf83159c606289adbf in term 0.
I20260811 10:12:11.293982 11341 leader_election.cc:302] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4169d06948ac41cf83159c606289adbf, 8d7c3b5d18a94541a8956f3ff4edb45f; no voters:
I20260811 10:12:11.294122 12122 raft_consensus.cc:2781] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:11.294181 12122 raft_consensus.cc:482] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:11.294198 12122 raft_consensus.cc:3037] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.295244 12122 raft_consensus.cc:504] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.295398 12122 leader_election.cc:288] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [CANDIDATE]: Term 1 election: Requested vote from peers b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485), 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3:36227)
I20260811 10:12:11.295553 11983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929d648f40ab4836ae65972de69a5bed" candidate_uuid: "4169d06948ac41cf83159c606289adbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f"
I20260811 10:12:11.295635 11983 raft_consensus.cc:3037] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.295564 11718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "929d648f40ab4836ae65972de69a5bed" candidate_uuid: "4169d06948ac41cf83159c606289adbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5d4aaebecd64ce5877a368facd1e7c9"
I20260811 10:12:11.295665 11718 raft_consensus.cc:3037] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:11.296319 11718 raft_consensus.cc:2454] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4169d06948ac41cf83159c606289adbf in term 1.
I20260811 10:12:11.296453 11983 raft_consensus.cc:2454] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4169d06948ac41cf83159c606289adbf in term 1.
I20260811 10:12:11.296512 11338 leader_election.cc:302] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4169d06948ac41cf83159c606289adbf, b5d4aaebecd64ce5877a368facd1e7c9; no voters:
I20260811 10:12:11.296603 12122 raft_consensus.cc:2781] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:11.296741 12122 raft_consensus.cc:686] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [term 1 LEADER]: Becoming Leader. State: Replica: 4169d06948ac41cf83159c606289adbf, State: Running, Role: LEADER
I20260811 10:12:11.296829 12122 consensus_queue.cc:237] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } }
I20260811 10:12:11.297276 12181 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: TxnStatusTablet state changed. Reason: New leader 4169d06948ac41cf83159c606289adbf. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.297423 12181 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:11.297714 12184 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:12:11.297861 12184 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:12:11.297803 11169 catalog_manager.cc:5495] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 4169d06948ac41cf83159c606289adbf (127.8.238.1). New cstate: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:11.312944 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.309847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36637 (local address 127.0.0.1:40388)
0811 10:12:11.309934 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.309948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.310003 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.310224 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.310224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.310226 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:11.310270 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.310272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.311343 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.311344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.311980 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.311982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.312449 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.312450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.312496 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.312510 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.312800 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.312806 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.312844 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.312878 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:12:11.312955 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.309941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.8.238.1:36637)
0811 10:12:11.309991 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.309993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.310004 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:11.310124 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.310124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.310145 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.310213 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:11.310319 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.311260 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.312057 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.312364 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.312438 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.312598 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.312717 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.312811 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.312851 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.312873 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:11.314294 11718 raft_consensus.cc:1261] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Refusing update from remote peer 4169d06948ac41cf83159c606289adbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.314390 11983 raft_consensus.cc:1261] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Refusing update from remote peer 4169d06948ac41cf83159c606289adbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.314590 12181 consensus_queue.cc:1035] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.314733 12122 consensus_queue.cc:1035] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.315665 12123 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: TxnStatusTablet state changed. Reason: New leader 4169d06948ac41cf83159c606289adbf. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.315793 12123 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.315699 12145 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: TxnStatusTablet state changed. Reason: New leader 4169d06948ac41cf83159c606289adbf. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.315846 12145 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.316654 12122 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.316736 12122 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:11.316797 12134 mvcc.cc:205] Tried to move back new op lower bound from 7317271065861238784 to 7317271065792446464. Current Snapshot: MvccSnapshot[applied={T|T < 7317271065861238784 or (T in {7317271065861238784})}]
I20260811 10:12:11.317102 12133 mvcc.cc:205] Tried to move back new op lower bound from 7317271065861238784 to 7317271065792446464. Current Snapshot: MvccSnapshot[applied={T|T < 7317271065861238784 or (T in {7317271065861238784})}]
I20260811 10:12:11.317211 12145 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.317294 12145 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 8d7c3b5d18a94541a8956f3ff4edb45f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.318225 12122 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.318315 12122 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P 4169d06948ac41cf83159c606289adbf: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:11.318770 12123 tablet_replica.cc:439] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4169d06948ac41cf83159c606289adbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 } } peers { permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 } } peers { permanent_uuid: "8d7c3b5d18a94541a8956f3ff4edb45f" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36227 } } }
I20260811 10:12:11.318879 12123 tablet_replica.cc:442] T 929d648f40ab4836ae65972de69a5bed P b5d4aaebecd64ce5877a368facd1e7c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:11.323454 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.321169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:44203 (local address 127.8.238.1:36637)
0811 10:12:11.321212 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.321214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.321223 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:11.321288 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.321289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.321300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.321354 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:11.321503 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.321959 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.322721 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.322971 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.323040 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.323258 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.323285 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.323322 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.323388 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.323404 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:11.323513 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.321105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36637 (local address 127.8.238.3:44203)
0811 10:12:11.321182 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.321195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.321238 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.321358 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.321359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.321360 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:11.321406 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.321407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.322047 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.322048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.322630 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.322632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.323068 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.323069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.323144 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.323164 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.323336 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.323337 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.323386 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.323437 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 10:12:11.326124 11466 raft_consensus.cc:1261] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Refusing update from remote peer 8d7c3b5d18a94541a8956f3ff4edb45f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.326134 11718 raft_consensus.cc:1261] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Refusing update from remote peer 8d7c3b5d18a94541a8956f3ff4edb45f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:11.326361 12145 consensus_queue.cc:1035] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4169d06948ac41cf83159c606289adbf" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.326505 12120 consensus_queue.cc:1035] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5d4aaebecd64ce5877a368facd1e7c9" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:11.340059 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.336254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:44943 (local address 127.8.238.3:36227)
0811 10:12:11.336300 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.336302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.336467 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:12:11.336666 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.336667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.336679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.336713 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:11.337161 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.337734 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.339147 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.339449 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.339528 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.339716 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.339799 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.339873 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.339969 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.339993 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:11.340319 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.336225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36227 (local address 127.8.238.1:44943)
0811 10:12:11.336403 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:11.336415 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:11.336576 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:11.336785 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:11.336786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:11.336787 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:11.337066 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.337068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.337813 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.337814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.339054 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:11.339057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.339538 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.339540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:11.339605 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.339622 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:11.339884 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:11.339886 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:12:11.340166 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 10:12:11.340216 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260811 10:12:11.349485 11173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:11.349593 11173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:11.350221 11173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebbc0369ad6249f49ff7bbae1ca78bc0: Sending DeleteTablet for 3 replicas of tablet 03e83ba448c1440a83e5ce67b884dbde
I20260811 10:12:11.350455 11429 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e83ba448c1440a83e5ce67b884dbde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:40378
I20260811 10:12:11.350487 11948 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e83ba448c1440a83e5ce67b884dbde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57170
I20260811 10:12:11.350502 11681 tablet_service.cc:1514] Processing DeleteTablet for tablet 03e83ba448c1440a83e5ce67b884dbde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:11 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57520
I20260811 10:12:11.350589 12151 tablet_replica.cc:330] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: stopping tablet replica
I20260811 10:12:11.350630 12150 tablet_replica.cc:330] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: stopping tablet replica
I20260811 10:12:11.350628 12156 tablet_replica.cc:330] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: stopping tablet replica
I20260811 10:12:11.350646 12151 raft_consensus.cc:2229] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:11.350682 12150 raft_consensus.cc:2229] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:11.350682 12156 raft_consensus.cc:2229] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:11.350693 12151 raft_consensus.cc:2258] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.350718 12150 raft_consensus.cc:2258] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.350723 12156 raft_consensus.cc:2258] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:11.351046 12156 ts_tablet_manager.cc:1905] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.351045 12150 ts_tablet_manager.cc:1905] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.351109 12151 ts_tablet_manager.cc:1905] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:11.352986 12150 ts_tablet_manager.cc:1918] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:11.353012 12156 ts_tablet_manager.cc:1918] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:11.353060 12156 log.cc:1196] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/03e83ba448c1440a83e5ce67b884dbde
I20260811 10:12:11.353060 12150 log.cc:1196] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/03e83ba448c1440a83e5ce67b884dbde
I20260811 10:12:11.353139 12151 ts_tablet_manager.cc:1918] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:11.353191 12151 log.cc:1196] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/03e83ba448c1440a83e5ce67b884dbde
I20260811 10:12:11.353232 12156 ts_tablet_manager.cc:1939] T 03e83ba448c1440a83e5ce67b884dbde P b5d4aaebecd64ce5877a368facd1e7c9: Deleting consensus metadata
I20260811 10:12:11.353307 12150 ts_tablet_manager.cc:1939] T 03e83ba448c1440a83e5ce67b884dbde P 4169d06948ac41cf83159c606289adbf: Deleting consensus metadata
I20260811 10:12:11.353404 12151 ts_tablet_manager.cc:1939] T 03e83ba448c1440a83e5ce67b884dbde P 8d7c3b5d18a94541a8956f3ff4edb45f: Deleting consensus metadata
I20260811 10:12:11.353751 11139 catalog_manager.cc:4841] TS 4169d06948ac41cf83159c606289adbf (127.8.238.1:36637): tablet 03e83ba448c1440a83e5ce67b884dbde (table test-table [id=7b508b8bd2cf48d49085db4b71063a17]) successfully deleted
I20260811 10:12:11.353777 11141 catalog_manager.cc:4841] TS 8d7c3b5d18a94541a8956f3ff4edb45f (127.8.238.3:36227): tablet 03e83ba448c1440a83e5ce67b884dbde (table test-table [id=7b508b8bd2cf48d49085db4b71063a17]) successfully deleted
I20260811 10:12:11.353849 11139 catalog_manager.cc:4841] TS b5d4aaebecd64ce5877a368facd1e7c9 (127.8.238.2:41485): tablet 03e83ba448c1440a83e5ce67b884dbde (table test-table [id=7b508b8bd2cf48d49085db4b71063a17]) successfully deleted
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[11087](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443131, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:11.366146 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.352169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.8.238.1:36637)
0811 10:12:11.352216 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.352218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.355931 (+ 3713us) server_negotiation.cc:407] Connection header received
0811 10:12:11.355962 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.355962 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.355975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.356040 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:11.356706 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.357483 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.358327 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.358646 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.358716 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.362075 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:11.362085 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:11.362085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:11.362093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:11.364831 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.365183 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.365187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.365188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.365257 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.365441 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.365442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.365443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.365620 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:11.365776 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.365902 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.365989 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 10:12:11.366508 11539 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40394
I20260811 10:12:11.374087 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:11.367656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.8.238.62:33183)
0811 10:12:11.367706 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:11.367708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:11.367752 (+ 44us) server_negotiation.cc:407] Connection header received
0811 10:12:11.367818 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:11.367819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:11.367830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:11.367911 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:11.368370 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.369159 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.369822 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:11.370138 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:11.370211 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:11.371352 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:11.371354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:11.371354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:11.371362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:11.373048 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.373359 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.373360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.373361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.373424 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:11.373606 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:11.373607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:11.373608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:11.373736 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:11.373834 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:11.373906 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 10:12:11.373981 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 10:12:11.374282 11173 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46694
I20260811 10:12:11.374903 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11309
I20260811 10:12:11.387599 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11570
I20260811 10:12:11.399039 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11832
I20260811 10:12:11.408981 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11110
2026-08-11T10:12:11Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2589 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[12211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[12211](info): set up 2 sockets
Aug 11 10:12:11 dist-test-slave-7f01 krb5kdc[12211](info): commencing operation
krb5kdc: starting...
W20260811 10:12:12.693698 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:12 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:12.818568 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:38013
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:38013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:12.902279 12247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:12.902486 12247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:12.902509 12247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:12.903810 12247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:12.903851 12247 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:12:12.903868 12247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:12.903882 12247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:12.903930 12247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:12.903947 12247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:12.905416 12247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:38013
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:38013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:12.905663 12247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:12.905880 12247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:12.910941 12257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:12.910948 12256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:12.910952 12259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:12.911119 12247 server_base.cc:1034] running on GCE node
I20260811 10:12:12.912119 12247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:12.912380 12247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:12.913532 12247 hybrid_clock.cc:648] HybridClock initialized: now 1786443132913513 us; error 44 us; skew 500 ppm
Aug 11 10:12:12 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:12.916325 12247 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:12.916792 12247 webserver.cc:460] Webserver started at http://127.8.238.62:34263/ using document root <none> and password file <none>
I20260811 10:12:12.917064 12247 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:12.917135 12247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:12.917285 12247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:12.918325 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "b742d1d1bb8c44a38eb338611e240c90"
format_stamp: "Formatted at 2026-08-11 10:12:12 on dist-test-slave-7f01"
server_key: "5f09ce5e8c5db1ef8ea312b34d066abe"
server_key_iv: "e7395fbac33fca350c490f948e7ed0c3"
server_key_version: "encryptionkey@0"
I20260811 10:12:12.918696 12247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "b742d1d1bb8c44a38eb338611e240c90"
format_stamp: "Formatted at 2026-08-11 10:12:12 on dist-test-slave-7f01"
server_key: "5f09ce5e8c5db1ef8ea312b34d066abe"
server_key_iv: "e7395fbac33fca350c490f948e7ed0c3"
server_key_version: "encryptionkey@0"
I20260811 10:12:12.920120 12247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:12.920900 12272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:12.921159 12247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:12.921296 12247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "b742d1d1bb8c44a38eb338611e240c90"
format_stamp: "Formatted at 2026-08-11 10:12:12 on dist-test-slave-7f01"
server_key: "5f09ce5e8c5db1ef8ea312b34d066abe"
server_key_iv: "e7395fbac33fca350c490f948e7ed0c3"
server_key_version: "encryptionkey@0"
I20260811 10:12:12.921394 12247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:12.936216 12247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:12.938249 12247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:12.938427 12247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:12.944248 12247 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:38013
I20260811 10:12:12.944322 12379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:38013 every 8 connection(s)
I20260811 10:12:12.944654 12247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:12.945279 12382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:12.947618 12382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Bootstrap starting.
I20260811 10:12:12.948274 12382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:12.948582 12382 log.cc:824] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:12.949302 12382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: No bootstrap required, opened a new log
I20260811 10:12:12.951573 12382 raft_consensus.cc:348] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } }
I20260811 10:12:12.951738 12382 raft_consensus.cc:374] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:12.951836 12382 raft_consensus.cc:729] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b742d1d1bb8c44a38eb338611e240c90, State: Initialized, Role: FOLLOWER
I20260811 10:12:12.952005 12382 consensus_queue.cc:260] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } }
I20260811 10:12:12.952092 12382 raft_consensus.cc:388] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:12.952137 12382 raft_consensus.cc:482] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:12.952185 12382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:12.952714 12382 raft_consensus.cc:504] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } }
I20260811 10:12:12.952845 12382 leader_election.cc:302] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b742d1d1bb8c44a38eb338611e240c90; no voters:
I20260811 10:12:12.953022 12382 leader_election.cc:288] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:12.953064 12388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:12.953280 12382 sys_catalog.cc:564] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:12.953341 12388 raft_consensus.cc:686] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [term 1 LEADER]: Becoming Leader. State: Replica: b742d1d1bb8c44a38eb338611e240c90, State: Running, Role: LEADER
I20260811 10:12:12.953480 12388 consensus_queue.cc:237] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } }
I20260811 10:12:12.953763 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12247
I20260811 10:12:12.953856 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:12.953893 12392 sys_catalog.cc:455] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b742d1d1bb8c44a38eb338611e240c90. Latest consensus state: current_term: 1 leader_uuid: "b742d1d1bb8c44a38eb338611e240c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } } }
I20260811 10:12:12.953995 12392 sys_catalog.cc:458] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:12.954058 9144 external_mini_cluster.cc:1428] Setting key 7523e474a6779bc5a4893899672c4094
I20260811 10:12:12.953888 12389 sys_catalog.cc:455] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b742d1d1bb8c44a38eb338611e240c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b742d1d1bb8c44a38eb338611e240c90" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38013 } } }
I20260811 10:12:12.954170 12389 sys_catalog.cc:458] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:12.954216 12403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:12.954819 12403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:12.956557 12403 catalog_manager.cc:1347] Generated new cluster ID: 5ee9fd430d904ebc96548282eb11405e
I20260811 10:12:12.956622 12403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:12:12 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:12.963563 12403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:12.964298 12403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:12.967643 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:12.954706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.8.238.62:38013)
0811 10:12:12.955980 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:12.955990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:12.956032 (+ 42us) server_negotiation.cc:407] Connection header received
0811 10:12:12.956161 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:12.956168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:12.956224 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:12.956385 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:12.957243 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:12.958661 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:12.959790 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:12.960456 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:12.960573 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:12.963711 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:12.963715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:12.963715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:12.963727 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:12.965911 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:12.966354 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:12.966357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:12.966358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:12.966471 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:12.966692 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:12.966693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:12.966694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:12.966985 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:12.967154 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:12.967234 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 10:12:12.967394 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1203,"thread_start_us":58,"threads_started":1}
I20260811 10:12:12.972563 12403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Generated new TSK 0
I20260811 10:12:12.972747 12403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:13.003245 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:38013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:13.087898 12438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:13.088112 12438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:13.088137 12438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:13.089308 12438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:13.089351 12438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:13.089372 12438 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:12:13.089413 12438 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:13.089448 12438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:13.090925 12438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:13.091176 12438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:13.091439 12438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:13.096659 12445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.096684 12449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.096650 12447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:13.097049 12438 server_base.cc:1034] running on GCE node
I20260811 10:12:13.097252 12438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:13.097500 12438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:13.098658 12438 hybrid_clock.cc:648] HybridClock initialized: now 1786443133098638 us; error 38 us; skew 500 ppm
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:13.101410 12438 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:13.101842 12438 webserver.cc:460] Webserver started at http://127.8.238.1:45479/ using document root <none> and password file <none>
I20260811 10:12:13.102077 12438 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:13.102146 12438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:13.102299 12438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:13.103348 12438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "453488ba38754d108c973672c72f37e2"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "403349128e4e421d9dbb3450d9152a4f"
server_key_iv: "7c2aa2f42fd564f093608ea3cda2fd0a"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.103793 12438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "453488ba38754d108c973672c72f37e2"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "403349128e4e421d9dbb3450d9152a4f"
server_key_iv: "7c2aa2f42fd564f093608ea3cda2fd0a"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.105198 12438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:13.105985 12465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.106169 12438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:13.106276 12438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "453488ba38754d108c973672c72f37e2"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "403349128e4e421d9dbb3450d9152a4f"
server_key_iv: "7c2aa2f42fd564f093608ea3cda2fd0a"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.106371 12438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:13.133549 12438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:13.135653 12438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:13.135869 12438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:13.136453 12438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:13.136518 12438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.136569 12438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:13.136602 12438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:13.144233 12438 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:33003
I20260811 10:12:13.144641 12438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:13.145062 12657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:33003 every 8 connection(s)
I20260811 10:12:13.147462 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.137059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:53113 (local address 127.8.238.62:38013)
0811 10:12:13.137118 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.137120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.137389 (+ 269us) server_negotiation.cc:407] Connection header received
0811 10:12:13.137997 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.137999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.138019 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.138077 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.139138 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.140002 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.141707 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.142064 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.142143 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.144734 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.144736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.144737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.144744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.146137 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.146589 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.146590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.146590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.146647 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.146900 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.146900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.146901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.147007 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.147058 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.147309 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.147396 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:12:13.147541 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.137047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.1:53113)
0811 10:12:13.137250 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.137287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.137851 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.138205 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.138209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.138646 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.138992 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.139002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.140082 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.140083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.141595 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.141600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.142162 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.142163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.142317 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.142812 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.142820 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.144605 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.146294 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.146300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.146302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.146520 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.146708 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.146708 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.146709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.146818 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.147064 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.147069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.147189 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.147269 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.147389 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":111,"threads_started":1}
I20260811 10:12:13.148221 12658 heartbeater.cc:347] Connected to a master server at 127.8.238.62:38013
I20260811 10:12:13.148299 12658 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:13.148483 12658 heartbeater.cc:511] Master 127.8.238.62:38013 requested a full tablet report, sending...
I20260811 10:12:13.149111 12294 ts_manager.cc:194] Registered new tserver with Master: 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003)
I20260811 10:12:13.149140 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12438
I20260811 10:12:13.149255 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:13.149472 9144 external_mini_cluster.cc:1428] Setting key 6a196338a4646837b7911e7af33f0065
I20260811 10:12:13.150070 12294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:53113
I20260811 10:12:13.155887 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.149352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.1:58883)
0811 10:12:13.149508 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.149521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.149587 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.149757 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.149758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.149950 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.150038 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.150040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.150930 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.150931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.151425 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.151427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.151954 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.151955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.152040 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.152431 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.152437 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.152880 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.154682 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.154683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.154683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.154922 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.155146 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.155146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.155146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.155203 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.155572 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.155573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.155618 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.155666 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.155796 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260811 10:12:13.155946 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.149417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:58883 (local address 127.8.238.62:38013)
0811 10:12:13.149453 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.149455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.149589 (+ 134us) server_negotiation.cc:407] Connection header received
0811 10:12:13.149646 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.149647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.149657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.149713 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.150143 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.150809 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.151509 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.151848 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.151920 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.152976 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.152977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.152977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.152983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.154604 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.155004 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.155005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.155006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.155074 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.155346 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.155348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.155348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.155496 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.155568 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.155723 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.155829 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:13.179486 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:38013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:13.263196 12674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:13.263386 12674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:13.263410 12674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:13.264629 12674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:13.264673 12674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:13.264694 12674 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:12:13.264706 12674 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:13.264767 12674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:13.266199 12674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:13.266443 12674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:13.266701 12674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:13.272032 12687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.272039 12689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.272055 12686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:13.272368 12674 server_base.cc:1034] running on GCE node
I20260811 10:12:13.272603 12674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:13.272852 12674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:13.274047 12674 hybrid_clock.cc:648] HybridClock initialized: now 1786443133274030 us; error 30 us; skew 500 ppm
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:13.276660 12674 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:13.277177 12674 webserver.cc:460] Webserver started at http://127.8.238.2:36561/ using document root <none> and password file <none>
I20260811 10:12:13.277384 12674 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:13.277426 12674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:13.277514 12674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:13.278445 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "b574d070a5484dc79b0b805e85c5834d"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "2f34980c0f93865cab733600e9764bca"
server_key_iv: "b443199cfefbd9a1981b101c5d897212"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.278743 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "b574d070a5484dc79b0b805e85c5834d"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "2f34980c0f93865cab733600e9764bca"
server_key_iv: "b443199cfefbd9a1981b101c5d897212"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.280126 12674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:13.280946 12701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.281155 12674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:13.281282 12674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "b574d070a5484dc79b0b805e85c5834d"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "2f34980c0f93865cab733600e9764bca"
server_key_iv: "b443199cfefbd9a1981b101c5d897212"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.281370 12674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:13.289798 12674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:13.291913 12674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:13.292076 12674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:13.292642 12674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:13.292703 12674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.292744 12674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:13.292773 12674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:13.301108 12674 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:46573
I20260811 10:12:13.301219 12900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:46573 every 8 connection(s)
I20260811 10:12:13.301535 12674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:13.302584 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.293083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:53889 (local address 127.8.238.62:38013)
0811 10:12:13.293143 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.293147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.293360 (+ 213us) server_negotiation.cc:407] Connection header received
0811 10:12:13.293904 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.293906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.293923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.294004 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.294713 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.295259 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.296370 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.296648 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.296734 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.299116 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.299118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.299118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.299126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.301059 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.301559 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.301560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.301560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.301647 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.301967 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.301968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.301968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.302123 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.302176 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.302449 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.302520 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:13.302824 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.293048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.2:53889)
0811 10:12:13.293227 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.293262 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.293791 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.294066 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.294070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.294378 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.294611 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.294616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.295364 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.295365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.296262 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.296264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.296746 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.296747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.296911 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.297393 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.297402 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.299007 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.301160 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.301167 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.301168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.301463 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.301728 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.301729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.301730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.301878 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.302203 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.302208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.302323 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.302414 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.302534 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260811 10:12:13.303318 12901 heartbeater.cc:347] Connected to a master server at 127.8.238.62:38013
I20260811 10:12:13.303407 12901 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:13.303591 12901 heartbeater.cc:511] Master 127.8.238.62:38013 requested a full tablet report, sending...
I20260811 10:12:13.304052 12296 ts_manager.cc:194] Registered new tserver with Master: b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573)
I20260811 10:12:13.304471 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12674
I20260811 10:12:13.304548 12296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:53889
I20260811 10:12:13.304580 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:13.304805 9144 external_mini_cluster.cc:1428] Setting key 051eb22625b9ac7681591c2ac35c61e0
I20260811 10:12:13.311087 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.304556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.2:43701)
0811 10:12:13.304716 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.304727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.304778 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.305186 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.305188 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.305409 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.305489 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.305492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.306202 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.306203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.306683 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.306684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.307196 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.307197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.307291 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.307761 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.307766 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.308105 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.309904 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.309905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.309905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.310071 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.310310 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.310311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.310311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.310382 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.310764 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.310765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.310835 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.310931 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.311016 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260811 10:12:13.311194 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.305005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43701 (local address 127.8.238.62:38013)
0811 10:12:13.305062 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.305064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.305072 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:13.305095 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.305096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.305106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.305162 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.305587 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.306107 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.306778 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.307095 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.307170 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.308190 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.308192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.308192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.308200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.309799 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.310139 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.310140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.310140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.310207 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.310454 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.310455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.310456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.310686 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.310768 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.310985 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.311070 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:13.334479 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:38013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:13.417516 12913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:13.417721 12913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:13.417747 12913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:13.418829 12913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:13.418865 12913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:13.418884 12913 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 10:12:13.418896 12913 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:13.418913 12913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:13.420346 12913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:38013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:13.420523 12913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:13.420689 12913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:13.425590 12922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.425611 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:13.425714 12920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:13.425827 12913 server_base.cc:1034] running on GCE node
I20260811 10:12:13.426028 12913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:13.426256 12913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:13.427428 12913 hybrid_clock.cc:648] HybridClock initialized: now 1786443133427423 us; error 37 us; skew 500 ppm
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:13.430052 12913 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:13.430461 12913 webserver.cc:460] Webserver started at http://127.8.238.3:34609/ using document root <none> and password file <none>
I20260811 10:12:13.430680 12913 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:13.430747 12913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:13.430897 12913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:13.431969 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "8afcdc80051d4d07b66b7bf4362d5690"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "42ead8a55cfc5e3ccfedbc9fa195f688"
server_key_iv: "5b4960fa6aee7c4cf250a0a3e7995acf"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.432323 12913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "8afcdc80051d4d07b66b7bf4362d5690"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "42ead8a55cfc5e3ccfedbc9fa195f688"
server_key_iv: "5b4960fa6aee7c4cf250a0a3e7995acf"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.433637 12913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:13.434360 12939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.434536 12913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:13.434644 12913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "8afcdc80051d4d07b66b7bf4362d5690"
format_stamp: "Formatted at 2026-08-11 10:12:13 on dist-test-slave-7f01"
server_key: "42ead8a55cfc5e3ccfedbc9fa195f688"
server_key_iv: "5b4960fa6aee7c4cf250a0a3e7995acf"
server_key_version: "encryptionkey@0"
I20260811 10:12:13.434716 12913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:13.477029 12913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:13.479521 12913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:13.479677 12913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:13.480218 12913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:13.480284 12913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.480331 12913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:13.480356 12913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:13.487903 12913 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43147
I20260811 10:12:13.488034 13147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43147 every 8 connection(s)
I20260811 10:12:13.488306 12913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:13.489562 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12913
I20260811 10:12:13.489635 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:13.489840 9144 external_mini_cluster.cc:1428] Setting key 68c0f28f76d67416e5c796b58bbfdca2
I20260811 10:12:13.491115 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.480755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:54947 (local address 127.8.238.62:38013)
0811 10:12:13.480829 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.480832 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.481013 (+ 181us) server_negotiation.cc:407] Connection header received
0811 10:12:13.481516 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.481517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.481533 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.481613 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.482531 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.483386 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.484667 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.484915 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.485006 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.487965 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.487967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.487967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.487976 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.489636 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.490115 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.490117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.490117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.490183 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.490483 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.490484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.490485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.490621 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.490707 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.490981 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.491052 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:12:13.491273 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.480713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.3:54947)
0811 10:12:13.480876 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.480924 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.481417 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.481685 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.481709 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.482120 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.482420 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.482428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.483494 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.483495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.484575 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.484578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.485038 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.485039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.485196 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.485773 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.485783 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.487859 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.489732 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.489741 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.489748 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.490044 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.490260 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.490262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.490262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.490409 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.490695 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.490701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.490864 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.490953 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.491079 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260811 10:12:13.491698 13148 heartbeater.cc:347] Connected to a master server at 127.8.238.62:38013
I20260811 10:12:13.491824 13148 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:13.492007 13148 heartbeater.cc:511] Master 127.8.238.62:38013 requested a full tablet report, sending...
I20260811 10:12:13.492359 12296 ts_manager.cc:194] Registered new tserver with Master: 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:13.492868 12296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:54947
I20260811 10:12:13.497763 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.492723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.8.238.3:39789)
0811 10:12:13.492867 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.492879 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.492949 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.493107 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.493109 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.493310 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.493368 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.493370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.493993 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.493994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.494463 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.494464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.494762 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.494763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.494821 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.495150 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.495154 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.495407 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.496979 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.496980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.496980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.497147 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.497326 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.497327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.497327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.497378 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.497544 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.497544 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.497578 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.497628 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.497702 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260811 10:12:13.497781 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.492814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:39789 (local address 127.8.238.62:38013)
0811 10:12:13.492852 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.492854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.492934 (+ 80us) server_negotiation.cc:407] Connection header received
0811 10:12:13.493010 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.493011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.493020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.493086 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.493450 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.493894 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.494531 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.494702 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.494748 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.495481 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.495483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.495483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.495489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.496901 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.497217 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.497218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.497218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.497268 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.497415 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.497416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.497416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.497496 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.497540 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.497641 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.497697 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:13.501492 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:13.526237 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.519298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.8.238.62:38013)
0811 10:12:13.519376 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.519378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.519386 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:13.519413 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.519414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.519424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.519476 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.520105 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.520605 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.521306 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.521505 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.521563 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.523859 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.523860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.523860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.523886 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.525154 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.525464 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.525465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.525465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.525544 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.525741 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.525741 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.525741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.525884 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.525981 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.526111 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.526172 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:12:13.528009 12296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:13.529671 12296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:13.536561 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33003 (local address 127.0.0.1:48654)
0811 10:12:13.532312 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.532355 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.532425 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.532835 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.532841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.532844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.532936 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.532939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534285 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.534286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.535251 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.535254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536239 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.536324 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536338 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.536420 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.536477 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260811 10:12:13.536661 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:46573 (local address 127.0.0.1:34328)
0811 10:12:13.532422 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.532432 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.532475 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.532834 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.532838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.532841 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.532951 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.532954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534258 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.534259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.535454 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.535458 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536366 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.536444 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536448 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.536506 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.536548 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":37,"thread_start_us":96,"threads_started":1}
I20260811 10:12:13.536660 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.8.238.2:46573)
0811 10:12:13.532665 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.532669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.532679 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:13.532717 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.532720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.532734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.532815 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.533019 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534156 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.535546 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536280 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536352 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536407 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.536519 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.536555 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260811 10:12:13.536989 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.0.0.1:60800)
0811 10:12:13.532663 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.532673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.533030 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.533227 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.533227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.533230 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.533278 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.533280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534095 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.534097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.535456 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.535458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536541 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.536605 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536609 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.536874 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.536918 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":33,"thread_start_us":85,"threads_started":1}
I20260811 10:12:13.537257 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.8.238.1:33003)
0811 10:12:13.532606 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.532612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.532622 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:13.532673 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.532675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.532693 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.533026 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.533098 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534181 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.535363 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536143 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536227 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536289 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.537016 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.537142 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":95,"threads_started":1}
I20260811 10:12:13.537309 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.532440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.8.238.3:43147)
0811 10:12:13.532946 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.532951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.532961 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:13.533111 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.533114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.533130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.533231 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.533335 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.534006 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.535815 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.536389 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.536450 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.536505 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.536902 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.537176 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":124,"threads_started":1}
I20260811 10:12:13.537389 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 85bf5ed17572426d9f08953097e52b0c (DEFAULT_TABLE table=test-table [id=2cf52122da5b484bbcda9d006140f414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.537804 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bf5ed17572426d9f08953097e52b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.537897 12545 tablet_service.cc:1467] Processing CreateTablet for tablet 85bf5ed17572426d9f08953097e52b0c (DEFAULT_TABLE table=test-table [id=2cf52122da5b484bbcda9d006140f414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.538255 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bf5ed17572426d9f08953097e52b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.540472 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 85bf5ed17572426d9f08953097e52b0c (DEFAULT_TABLE table=test-table [id=2cf52122da5b484bbcda9d006140f414]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.540812 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85bf5ed17572426d9f08953097e52b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.542156 13189 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Bootstrap starting.
I20260811 10:12:13.542725 13189 tablet_bootstrap.cc:654] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.543072 13189 log.cc:824] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:13.544000 13189 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: No bootstrap required, opened a new log
I20260811 10:12:13.543999 13191 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Bootstrap starting.
I20260811 10:12:13.544073 13189 ts_tablet_manager.cc:1397] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:13.544997 13191 tablet_bootstrap.cc:654] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.545394 13191 log.cc:824] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:13.545768 13189 raft_consensus.cc:348] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.545930 13194 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Bootstrap starting.
I20260811 10:12:13.545964 13189 raft_consensus.cc:374] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.546008 13189 raft_consensus.cc:729] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453488ba38754d108c973672c72f37e2, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.546142 13189 consensus_queue.cc:260] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.546353 13191 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: No bootstrap required, opened a new log
I20260811 10:12:13.546423 13191 ts_tablet_manager.cc:1397] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:13.546430 13189 ts_tablet_manager.cc:1428] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 10:12:13.546558 12658 heartbeater.cc:503] Master 127.8.238.62:38013 was elected leader, sending a full tablet report...
I20260811 10:12:13.546582 13194 tablet_bootstrap.cc:654] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.546921 13194 log.cc:824] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:13.547689 13194 tablet_bootstrap.cc:492] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: No bootstrap required, opened a new log
I20260811 10:12:13.547986 13194 ts_tablet_manager.cc:1397] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:13.548602 13191 raft_consensus.cc:348] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.548789 13191 raft_consensus.cc:374] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.548837 13191 raft_consensus.cc:729] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.548944 13191 consensus_queue.cc:260] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.549252 13191 ts_tablet_manager.cc:1428] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:13.549325 13148 heartbeater.cc:503] Master 127.8.238.62:38013 was elected leader, sending a full tablet report...
I20260811 10:12:13.549825 13194 raft_consensus.cc:348] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.549998 13194 raft_consensus.cc:374] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.550045 13194 raft_consensus.cc:729] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.550155 13194 consensus_queue.cc:260] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.550453 13194 ts_tablet_manager.cc:1428] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:13.550566 12901 heartbeater.cc:503] Master 127.8.238.62:38013 was elected leader, sending a full tablet report...
W20260811 10:12:13.552099 12902 tablet.cc:2369] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:12:13.645457 12659 tablet.cc:2369] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:13.685770 13197 raft_consensus.cc:482] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:13.685918 13197 raft_consensus.cc:504] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.686378 13197 leader_election.cc:288] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:13.689419 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.686736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:45919 (local address 127.8.238.3:43147)
0811 10:12:13.686852 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.686854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.686863 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:13.686901 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.686902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.686912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.686983 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.687179 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.687677 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.688681 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.689048 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.689135 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.689144 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.689293 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.689334 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 10:12:13.689422 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.686647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:37721 (local address 127.8.238.2:46573)
0811 10:12:13.686703 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.686705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.686793 (+ 88us) server_negotiation.cc:407] Connection header received
0811 10:12:13.686944 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.686945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.686959 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.687034 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.687195 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.687686 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.688680 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.689050 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.689132 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.689144 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.689296 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.689337 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:12:13.689440 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.686548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:46573 (local address 127.8.238.1:37721)
0811 10:12:13.686702 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.686715 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.686829 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.687039 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.687040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.687042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.687118 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.687120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.687827 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.687828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.688582 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.688584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.689146 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.689147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.689220 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.689224 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.689289 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.689335 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260811 10:12:13.689455 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.686578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.8.238.1:45919)
0811 10:12:13.686704 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.686715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.686798 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.686990 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.686991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.686993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.687081 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.687084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.687790 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.687791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.688585 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.688587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.689139 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.689140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.689215 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.689219 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.689287 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.689337 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260811 10:12:13.689764 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bf5ed17572426d9f08953097e52b0c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d" is_pre_election: true
I20260811 10:12:13.689864 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bf5ed17572426d9f08953097e52b0c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690" is_pre_election: true
I20260811 10:12:13.689992 12816 raft_consensus.cc:2454] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:13.690025 13065 raft_consensus.cc:2454] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:13.690291 12467 leader_election.cc:302] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:13.690438 13197 raft_consensus.cc:2781] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:13.690503 13197 raft_consensus.cc:482] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:13.690521 13197 raft_consensus.cc:3037] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.691260 13197 raft_consensus.cc:504] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.691411 13197 leader_election.cc:288] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Requested vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:13.691581 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bf5ed17572426d9f08953097e52b0c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690"
I20260811 10:12:13.691592 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85bf5ed17572426d9f08953097e52b0c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d"
I20260811 10:12:13.691665 13065 raft_consensus.cc:3037] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.691665 12816 raft_consensus.cc:3037] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.692628 12816 raft_consensus.cc:2454] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:13.692628 13065 raft_consensus.cc:2454] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:13.692807 12467 leader_election.cc:302] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:13.692910 13197 raft_consensus.cc:2781] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:13.693059 13197 raft_consensus.cc:686] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Becoming Leader. State: Replica: 453488ba38754d108c973672c72f37e2, State: Running, Role: LEADER
I20260811 10:12:13.693176 13197 consensus_queue.cc:237] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.694032 12294 catalog_manager.cc:5495] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 453488ba38754d108c973672c72f37e2 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "453488ba38754d108c973672c72f37e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } health_report { overall_health: UNKNOWN } } }
Aug 11 10:12:13 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443132, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:13.735210 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.728445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:38013 (local address 127.0.0.1:38772)
0811 10:12:13.728668 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.728679 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.728749 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.728884 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.728884 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.729205 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:13.729260 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.729263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.729934 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.729935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.730463 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.730464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.730796 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.730796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.730838 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.731436 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:13.731440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:13.732829 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:13.734251 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.734258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.734261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.734462 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.734646 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:13.734646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:13.734647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:13.734704 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:13.734932 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:13.734938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:13.734978 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.735030 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.735118 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":122,"threads_started":1}
I20260811 10:12:13.735291 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.728509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.8.238.62:38013)
0811 10:12:13.728583 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.728586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.728744 (+ 158us) server_negotiation.cc:407] Connection header received
0811 10:12:13.728797 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.728798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.728807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.728859 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:13.729349 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.729861 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.730539 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.730725 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.730784 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.732908 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:13.732909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:13.732909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:13.732915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:13.734177 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.734528 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.734528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.734529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.734586 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:13.734756 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:13.734757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:13.734757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:13.734876 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:13.734928 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.735063 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.735165 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":39}
I20260811 10:12:13.737444 12294 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:38772:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:12:13.737709 12294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 10:12:13.738921 13149 tablet.cc:2369] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:13.739387 12545 tablet_service.cc:1467] Processing CreateTablet for tablet 0a8fa709e6bc46d29b51ace152e75523 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=578333ad9f494a9a998d2bb1093bb091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:13.739421 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 0a8fa709e6bc46d29b51ace152e75523 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=578333ad9f494a9a998d2bb1093bb091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:13.739601 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a8fa709e6bc46d29b51ace152e75523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.739598 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a8fa709e6bc46d29b51ace152e75523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.740983 13194 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: Bootstrap starting.
I20260811 10:12:13.741081 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 0a8fa709e6bc46d29b51ace152e75523 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=578333ad9f494a9a998d2bb1093bb091]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:13.741281 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a8fa709e6bc46d29b51ace152e75523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.741638 13194 tablet_bootstrap.cc:654] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.742426 13194 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: No bootstrap required, opened a new log
I20260811 10:12:13.742472 13194 ts_tablet_manager.cc:1397] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:13.742604 13194 raft_consensus.cc:348] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.742651 13194 raft_consensus.cc:374] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.742664 13194 raft_consensus.cc:729] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.742738 13194 consensus_queue.cc:260] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.742838 13194 ts_tablet_manager.cc:1428] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.742909 13189 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: Bootstrap starting.
I20260811 10:12:13.742892 13201 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.742977 13201 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.743399 13189 tablet_bootstrap.cc:654] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.743810 13191 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: Bootstrap starting.
I20260811 10:12:13.744375 13191 tablet_bootstrap.cc:654] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.744496 13189 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: No bootstrap required, opened a new log
I20260811 10:12:13.744555 13189 ts_tablet_manager.cc:1397] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:12:13.744704 13189 raft_consensus.cc:348] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.744782 13189 raft_consensus.cc:374] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.744803 13189 raft_consensus.cc:729] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453488ba38754d108c973672c72f37e2, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.744854 13189 consensus_queue.cc:260] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.744947 13189 ts_tablet_manager.cc:1428] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.745009 13197 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.745123 13197 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.745124 13191 tablet_bootstrap.cc:492] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: No bootstrap required, opened a new log
I20260811 10:12:13.745172 13191 ts_tablet_manager.cc:1397] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:13.745298 13191 raft_consensus.cc:348] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.745352 13191 raft_consensus.cc:374] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.745366 13191 raft_consensus.cc:729] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.745438 13191 consensus_queue.cc:260] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.745573 13191 ts_tablet_manager.cc:1428] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.745626 13199 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.745707 13199 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.789007 13199 raft_consensus.cc:482] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:13.789135 13199 raft_consensus.cc:504] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.789558 13199 leader_election.cc:288] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003), b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573)
I20260811 10:12:13.793574 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.789821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:38535 (local address 127.8.238.1:33003)
0811 10:12:13.789889 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.789891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.789904 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:12:13.789944 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.789944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.789956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.790015 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.790291 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.791060 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.792594 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.793183 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.793264 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.793275 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.793457 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.793504 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:12:13.793574 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.789869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:45303 (local address 127.8.238.2:46573)
0811 10:12:13.789915 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.789918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.790185 (+ 267us) server_negotiation.cc:407] Connection header received
0811 10:12:13.790231 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.790231 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.790255 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.790317 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:13.790465 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.791172 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.792594 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.793185 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.793264 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.793275 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.793457 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.793500 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:13.793612 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.789728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33003 (local address 127.8.238.3:38535)
0811 10:12:13.789817 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.789831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.789891 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.790079 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.790088 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.790091 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.790204 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.790206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.791140 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.791141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.792480 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.792485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.793273 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.793274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.793368 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.793373 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.793434 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.793486 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":30}
I20260811 10:12:13.793624 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.789750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:46573 (local address 127.8.238.3:45303)
0811 10:12:13.789940 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.789948 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.790179 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.790333 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.790334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.790336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:13.790396 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.790397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.791240 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.791240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.792481 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.792485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.793290 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.793291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.793371 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.793373 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.793434 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.793486 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":7,"thread_start_us":106,"threads_started":1}
I20260811 10:12:13.794075 12583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a8fa709e6bc46d29b51ace152e75523" candidate_uuid: "8afcdc80051d4d07b66b7bf4362d5690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "453488ba38754d108c973672c72f37e2" is_pre_election: true
I20260811 10:12:13.794118 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a8fa709e6bc46d29b51ace152e75523" candidate_uuid: "8afcdc80051d4d07b66b7bf4362d5690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d" is_pre_election: true
I20260811 10:12:13.794188 12583 raft_consensus.cc:2454] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8afcdc80051d4d07b66b7bf4362d5690 in term 0.
I20260811 10:12:13.794229 12816 raft_consensus.cc:2454] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8afcdc80051d4d07b66b7bf4362d5690 in term 0.
I20260811 10:12:13.794446 12949 leader_election.cc:302] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, 8afcdc80051d4d07b66b7bf4362d5690; no voters:
I20260811 10:12:13.794598 13199 raft_consensus.cc:2781] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:13.794644 13199 raft_consensus.cc:482] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:13.794669 13199 raft_consensus.cc:3037] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.795439 13199 raft_consensus.cc:504] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.795593 13199 leader_election.cc:288] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [CANDIDATE]: Term 1 election: Requested vote from peers 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003), b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573)
I20260811 10:12:13.795797 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a8fa709e6bc46d29b51ace152e75523" candidate_uuid: "8afcdc80051d4d07b66b7bf4362d5690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d"
I20260811 10:12:13.795799 12583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a8fa709e6bc46d29b51ace152e75523" candidate_uuid: "8afcdc80051d4d07b66b7bf4362d5690" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "453488ba38754d108c973672c72f37e2"
I20260811 10:12:13.795885 12816 raft_consensus.cc:3037] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.795884 12583 raft_consensus.cc:3037] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.796701 12816 raft_consensus.cc:2454] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8afcdc80051d4d07b66b7bf4362d5690 in term 1.
I20260811 10:12:13.796756 12583 raft_consensus.cc:2454] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8afcdc80051d4d07b66b7bf4362d5690 in term 1.
I20260811 10:12:13.796948 12948 leader_election.cc:302] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8afcdc80051d4d07b66b7bf4362d5690, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:13.797081 13199 raft_consensus.cc:2781] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:13.797281 13199 raft_consensus.cc:686] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 LEADER]: Becoming Leader. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Running, Role: LEADER
I20260811 10:12:13.797403 13199 consensus_queue.cc:237] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.797923 13226 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: TxnStatusTablet state changed. Reason: New leader 8afcdc80051d4d07b66b7bf4362d5690. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.798024 13226 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:13.798228 13231 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:12:13.798352 13231 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:12:13.798290 12294 catalog_manager.cc:5495] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3). New cstate: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } health_report { overall_health: HEALTHY } } }
I20260811 10:12:13.818540 13214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.815884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.0.0.1:60808)
0811 10:12:13.815968 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.815982 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.816037 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.816229 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.816231 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.816232 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:13.816302 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.816305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.816987 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.816988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.817571 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.817573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.817954 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.817955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.818034 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.818055 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.818329 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.818341 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.818408 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.818462 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:12:13.818531 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.815962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.8.238.3:43147)
0811 10:12:13.816012 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.816014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.816025 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:13.816120 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.816121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.816140 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.816222 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:13.816383 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.816890 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.817658 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.817873 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.817953 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.818150 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.818255 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.818351 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.818411 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.818436 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:12:13.820600 12816 raft_consensus.cc:1261] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Refusing update from remote peer 8afcdc80051d4d07b66b7bf4362d5690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.820621 12583 raft_consensus.cc:1261] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Refusing update from remote peer 8afcdc80051d4d07b66b7bf4362d5690: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.820956 13199 consensus_queue.cc:1035] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:13.821076 13226 consensus_queue.cc:1035] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690 [LEADER]: Connected to new peer: Peer: permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:13.822448 13197 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: TxnStatusTablet state changed. Reason: New leader 8afcdc80051d4d07b66b7bf4362d5690. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.822571 13197 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.823217 13201 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: TxnStatusTablet state changed. Reason: New leader 8afcdc80051d4d07b66b7bf4362d5690. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.823323 13201 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.824260 13226 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.824364 13226 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:13.824429 13226 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.824473 13226 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 8afcdc80051d4d07b66b7bf4362d5690: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:13.824512 13201 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.824589 13201 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P b574d070a5484dc79b0b805e85c5834d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.824540 13197 tablet_replica.cc:439] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8afcdc80051d4d07b66b7bf4362d5690" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } } }
I20260811 10:12:13.824620 13197 tablet_replica.cc:442] T 0a8fa709e6bc46d29b51ace152e75523 P 453488ba38754d108c973672c72f37e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:13.825100 13239 mvcc.cc:205] Tried to move back new op lower bound from 7317271076126425088 to 7317271076034797568. Current Snapshot: MvccSnapshot[applied={T|T < 7317271076126425088}]
I20260811 10:12:13.825261 13242 mvcc.cc:205] Tried to move back new op lower bound from 7317271076126425088 to 7317271076034797568. Current Snapshot: MvccSnapshot[applied={T|T < 7317271076126425088 or (T in {7317271076126425088})}]
I20260811 10:12:13.830096 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.827646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48666 (local address 127.8.238.1:33003)
0811 10:12:13.827692 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.827694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.827703 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:13.827836 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.827837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.827853 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.827927 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:13.828072 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.828535 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.829312 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.829514 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.829594 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.829780 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.829873 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.829966 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.829996 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.830018 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:12:13.835557 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.832942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:56857 (local address 127.8.238.3:43147)
0811 10:12:13.832985 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.832987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.833077 (+ 90us) server_negotiation.cc:407] Connection header received
0811 10:12:13.833129 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.833130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.833148 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.833190 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:13.833314 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.833747 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.834900 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.835075 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.835138 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.835339 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.835364 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.835412 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.835494 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.835510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:13.835637 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.832822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.8.238.1:56857)
0811 10:12:13.832996 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.833010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.833076 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.833203 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.833204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.833205 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:13.833261 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.833263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.833817 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.833818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.834794 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.834798 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.835157 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.835158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.835225 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.835242 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.835435 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.835438 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.835493 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.835541 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260811 10:12:13.838410 13065 raft_consensus.cc:1261] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.838582 12816 raft_consensus.cc:1261] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.838665 13197 consensus_queue.cc:1035] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:13.838784 13205 consensus_queue.cc:1035] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:13.840512 13236 mvcc.cc:205] Tried to move back new op lower bound from 7317271076200005632 to 7317271075607879680. Current Snapshot: MvccSnapshot[applied={T|T < 7317271076200005632}]
I20260811 10:12:13.847632 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.845077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:44541 (local address 127.8.238.1:33003)
0811 10:12:13.845113 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:13.845115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:13.845152 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:12:13.845215 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:13.845216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:13.845225 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:13.845279 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:13.845429 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.845876 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.846890 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.847189 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.847261 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.847413 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.847445 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.847498 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:13.847544 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 10:12:13.847563 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:13.847687 13155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:13.845044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33003 (local address 127.8.238.3:44541)
0811 10:12:13.845107 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:13.845120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:13.845167 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:13.845282 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:13.845282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:13.845283 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:13.845349 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.845350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.845965 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.845967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.846827 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:13.846830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:13.847267 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:13.847268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:13.847320 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:13.847333 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:13.847505 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:13.847507 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:12:13.847541 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 10:12:13.847577 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 10:12:13.856902 12296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:13.857010 12296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:13.857753 12296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Sending DeleteTablet for 3 replicas of tablet 85bf5ed17572426d9f08953097e52b0c
I20260811 10:12:13.858069 12545 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bf5ed17572426d9f08953097e52b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:13 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48654
I20260811 10:12:13.858112 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bf5ed17572426d9f08953097e52b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:13 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:60800
I20260811 10:12:13.858402 13267 tablet_replica.cc:330] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: stopping tablet replica
I20260811 10:12:13.858465 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet 85bf5ed17572426d9f08953097e52b0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:13 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34328
I20260811 10:12:13.858595 13269 tablet_replica.cc:330] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: stopping tablet replica
I20260811 10:12:13.858736 13267 raft_consensus.cc:2229] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:13.858875 13269 raft_consensus.cc:2229] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:13.858897 12296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:13.859058 12296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:13.859213 13269 raft_consensus.cc:2258] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:13.859349 13267 raft_consensus.cc:2258] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:13.860332 13267 ts_tablet_manager.cc:1905] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:13.860391 12545 tablet_service.cc:1467] Processing CreateTablet for tablet 7f801e5df936463ab4b55090f2c8665c (DEFAULT_TABLE table=test-table [id=19e76a9425dd4cd5ade541f2b703ed05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.860534 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f801e5df936463ab4b55090f2c8665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.861913 13189 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Bootstrap starting.
I20260811 10:12:13.862339 13189 tablet_bootstrap.cc:654] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.862521 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 7f801e5df936463ab4b55090f2c8665c (DEFAULT_TABLE table=test-table [id=19e76a9425dd4cd5ade541f2b703ed05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.862633 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f801e5df936463ab4b55090f2c8665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.862323 13267 ts_tablet_manager.cc:1918] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:13.862861 13267 log.cc:1196] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/85bf5ed17572426d9f08953097e52b0c
I20260811 10:12:13.863219 13267 ts_tablet_manager.cc:1939] T 85bf5ed17572426d9f08953097e52b0c P 453488ba38754d108c973672c72f37e2: Deleting consensus metadata
I20260811 10:12:13.863686 12275 catalog_manager.cc:4841] TS 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003): tablet 85bf5ed17572426d9f08953097e52b0c (table test-table [id=2cf52122da5b484bbcda9d006140f414]) successfully deleted
I20260811 10:12:13.863920 13191 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Bootstrap starting.
I20260811 10:12:13.863919 13271 tablet_replica.cc:330] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: stopping tablet replica
I20260811 10:12:13.864102 13271 raft_consensus.cc:2229] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:13.864221 13271 raft_consensus.cc:2258] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:13.864280 13191 tablet_bootstrap.cc:654] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.864727 13189 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: No bootstrap required, opened a new log
I20260811 10:12:13.864778 13189 ts_tablet_manager.cc:1397] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 10:12:13.864936 13189 raft_consensus.cc:348] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.865012 13189 raft_consensus.cc:374] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.865034 13189 raft_consensus.cc:729] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453488ba38754d108c973672c72f37e2, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.865077 13189 consensus_queue.cc:260] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.865170 13189 ts_tablet_manager.cc:1428] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.865167 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 7f801e5df936463ab4b55090f2c8665c (DEFAULT_TABLE table=test-table [id=19e76a9425dd4cd5ade541f2b703ed05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:13.865275 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f801e5df936463ab4b55090f2c8665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:13.866508 13194 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Bootstrap starting.
I20260811 10:12:13.866966 13194 tablet_bootstrap.cc:654] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:13.867139 13271 ts_tablet_manager.cc:1905] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:13.867185 13191 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: No bootstrap required, opened a new log
I20260811 10:12:13.867246 13191 ts_tablet_manager.cc:1397] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 10:12:13.867432 13191 raft_consensus.cc:348] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.867483 13269 ts_tablet_manager.cc:1905] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:13.867498 13191 raft_consensus.cc:374] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.868000 13191 raft_consensus.cc:729] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.868055 13191 consensus_queue.cc:260] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.868177 13191 ts_tablet_manager.cc:1428] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:13.869129 13269 ts_tablet_manager.cc:1918] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:13.869206 13269 log.cc:1196] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/85bf5ed17572426d9f08953097e52b0c
I20260811 10:12:13.869247 13194 tablet_bootstrap.cc:492] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: No bootstrap required, opened a new log
I20260811 10:12:13.869295 13194 ts_tablet_manager.cc:1397] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 10:12:13.869405 13271 ts_tablet_manager.cc:1918] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:13.869474 13271 log.cc:1196] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/85bf5ed17572426d9f08953097e52b0c
I20260811 10:12:13.869451 13194 raft_consensus.cc:348] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.869493 13269 ts_tablet_manager.cc:1939] T 85bf5ed17572426d9f08953097e52b0c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting consensus metadata
I20260811 10:12:13.869511 13194 raft_consensus.cc:374] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:13.869530 13194 raft_consensus.cc:729] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Initialized, Role: FOLLOWER
I20260811 10:12:13.869570 13194 consensus_queue.cc:260] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.869640 13194 ts_tablet_manager.cc:1428] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:13.869889 12273 catalog_manager.cc:4841] TS 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147): tablet 85bf5ed17572426d9f08953097e52b0c (table test-table [id=2cf52122da5b484bbcda9d006140f414]) successfully deleted
I20260811 10:12:13.870332 13271 ts_tablet_manager.cc:1939] T 85bf5ed17572426d9f08953097e52b0c P b574d070a5484dc79b0b805e85c5834d: Deleting consensus metadata
I20260811 10:12:13.870898 12274 catalog_manager.cc:4841] TS b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573): tablet 85bf5ed17572426d9f08953097e52b0c (table test-table [id=2cf52122da5b484bbcda9d006140f414]) successfully deleted
I20260811 10:12:13.930332 13264 raft_consensus.cc:482] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:13.930475 13264 raft_consensus.cc:504] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.930667 13264 leader_election.cc:288] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:13.930868 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f801e5df936463ab4b55090f2c8665c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d" is_pre_election: true
I20260811 10:12:13.930922 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f801e5df936463ab4b55090f2c8665c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690" is_pre_election: true
I20260811 10:12:13.930984 12816 raft_consensus.cc:2454] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:13.931023 13065 raft_consensus.cc:2454] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:13.931552 12466 leader_election.cc:302] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, 8afcdc80051d4d07b66b7bf4362d5690; no voters:
I20260811 10:12:13.931672 13264 raft_consensus.cc:2781] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:13.931710 13264 raft_consensus.cc:482] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:13.931756 13264 raft_consensus.cc:3037] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.932761 13264 raft_consensus.cc:504] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.932900 13264 leader_election.cc:288] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Requested vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:13.933055 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f801e5df936463ab4b55090f2c8665c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d"
I20260811 10:12:13.933126 12816 raft_consensus.cc:3037] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.933118 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f801e5df936463ab4b55090f2c8665c" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690"
I20260811 10:12:13.933197 13065 raft_consensus.cc:3037] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:13.934083 12816 raft_consensus.cc:2454] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:13.934087 13065 raft_consensus.cc:2454] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:13.934265 12467 leader_election.cc:302] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:13.934490 13264 raft_consensus.cc:2781] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:13.934558 13264 raft_consensus.cc:686] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Becoming Leader. State: Replica: 453488ba38754d108c973672c72f37e2, State: Running, Role: LEADER
I20260811 10:12:13.934610 13264 consensus_queue.cc:237] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:13.935243 12294 catalog_manager.cc:5495] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 453488ba38754d108c973672c72f37e2 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "453488ba38754d108c973672c72f37e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:13.952013 13065 raft_consensus.cc:1261] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.952282 13205 consensus_queue.cc:1035] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:13.952631 12816 raft_consensus.cc:1261] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:13.952855 13205 consensus_queue.cc:1035] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 10:12:13.959668 13239 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 10:12:15.480947 12294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:15.481057 12294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:15.481905 12294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Sending DeleteTablet for 3 replicas of tablet 7f801e5df936463ab4b55090f2c8665c
I20260811 10:12:15.482168 12545 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f801e5df936463ab4b55090f2c8665c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48654
I20260811 10:12:15.482203 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f801e5df936463ab4b55090f2c8665c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34328
I20260811 10:12:15.482401 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f801e5df936463ab4b55090f2c8665c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:60800
I20260811 10:12:15.482512 13312 tablet_replica.cc:330] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: stopping tablet replica
I20260811 10:12:15.482575 13312 raft_consensus.cc:2229] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:15.482612 13312 raft_consensus.cc:2258] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.482710 13311 tablet_replica.cc:330] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: stopping tablet replica
I20260811 10:12:15.482774 13311 raft_consensus.cc:2229] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:15.482863 13311 raft_consensus.cc:2258] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.483142 13311 ts_tablet_manager.cc:1905] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.485193 13311 ts_tablet_manager.cc:1918] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:12:15.485366 13311 log.cc:1196] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/7f801e5df936463ab4b55090f2c8665c
I20260811 10:12:15.485723 13311 ts_tablet_manager.cc:1939] T 7f801e5df936463ab4b55090f2c8665c P 453488ba38754d108c973672c72f37e2: Deleting consensus metadata
I20260811 10:12:15.486099 12275 catalog_manager.cc:4841] TS 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003): tablet 7f801e5df936463ab4b55090f2c8665c (table test-table [id=19e76a9425dd4cd5ade541f2b703ed05]) successfully deleted
I20260811 10:12:15.486152 13312 ts_tablet_manager.cc:1905] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.487828 13313 tablet_replica.cc:330] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: stopping tablet replica
I20260811 10:12:15.487983 13313 raft_consensus.cc:2229] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:15.488081 13313 raft_consensus.cc:2258] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.488384 13313 ts_tablet_manager.cc:1905] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.488533 13312 ts_tablet_manager.cc:1918] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:12:15.488677 13312 log.cc:1196] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/7f801e5df936463ab4b55090f2c8665c
I20260811 10:12:15.489010 13312 ts_tablet_manager.cc:1939] T 7f801e5df936463ab4b55090f2c8665c P b574d070a5484dc79b0b805e85c5834d: Deleting consensus metadata
I20260811 10:12:15.489403 12274 catalog_manager.cc:4841] TS b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573): tablet 7f801e5df936463ab4b55090f2c8665c (table test-table [id=19e76a9425dd4cd5ade541f2b703ed05]) successfully deleted
I20260811 10:12:15.490231 13313 ts_tablet_manager.cc:1918] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:12:15.490288 13313 log.cc:1196] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/7f801e5df936463ab4b55090f2c8665c
I20260811 10:12:15.490510 13313 ts_tablet_manager.cc:1939] T 7f801e5df936463ab4b55090f2c8665c P 8afcdc80051d4d07b66b7bf4362d5690: Deleting consensus metadata
I20260811 10:12:15.490900 12273 catalog_manager.cc:4841] TS 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147): tablet 7f801e5df936463ab4b55090f2c8665c (table test-table [id=19e76a9425dd4cd5ade541f2b703ed05]) successfully deleted
Aug 11 10:12:15 dist-test-slave-7f01 krb5kdc[12211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443135, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:12:15 dist-test-slave-7f01 krb5kdc[12211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443135, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:15.512995 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.503000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.8.238.62:38013)
0811 10:12:15.503153 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.503154 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.503207 (+ 53us) server_negotiation.cc:407] Connection header received
0811 10:12:15.503276 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.503276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.503287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.503382 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:15.504048 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.504573 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.505296 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.505494 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.505546 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.510552 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:15.510554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:15.510554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:15.510561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:15.511904 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:15.512218 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:15.512219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:15.512220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:15.512276 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:15.512445 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:15.512445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:15.512445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:15.512556 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:15.512638 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.512714 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.512812 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260811 10:12:15.514602 12294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:15.514760 12294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:15.516189 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 59ea420c484d427982c2589ad47149d7 (DEFAULT_TABLE table=test-table [id=7cced828741a419badfe09415e4af20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.516199 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 59ea420c484d427982c2589ad47149d7 (DEFAULT_TABLE table=test-table [id=7cced828741a419badfe09415e4af20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.516259 12545 tablet_service.cc:1467] Processing CreateTablet for tablet 59ea420c484d427982c2589ad47149d7 (DEFAULT_TABLE table=test-table [id=7cced828741a419badfe09415e4af20f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.516340 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ea420c484d427982c2589ad47149d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.516345 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ea420c484d427982c2589ad47149d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.516386 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ea420c484d427982c2589ad47149d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.517897 13332 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Bootstrap starting.
I20260811 10:12:15.517913 13331 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Bootstrap starting.
I20260811 10:12:15.518297 13332 tablet_bootstrap.cc:654] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.518301 13331 tablet_bootstrap.cc:654] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.518621 13335 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Bootstrap starting.
I20260811 10:12:15.518967 13332 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: No bootstrap required, opened a new log
I20260811 10:12:15.518982 13335 tablet_bootstrap.cc:654] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.519052 13332 ts_tablet_manager.cc:1397] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:15.519145 13331 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: No bootstrap required, opened a new log
I20260811 10:12:15.519214 13331 ts_tablet_manager.cc:1397] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:15.519225 13332 raft_consensus.cc:348] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.519381 13332 raft_consensus.cc:374] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.519361 13331 raft_consensus.cc:348] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.519428 13331 raft_consensus.cc:374] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.519464 13331 raft_consensus.cc:729] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.519487 13332 raft_consensus.cc:729] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.519517 13331 consensus_queue.cc:260] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.519579 13332 consensus_queue.cc:260] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.519742 13335 tablet_bootstrap.cc:492] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: No bootstrap required, opened a new log
I20260811 10:12:15.519812 13331 ts_tablet_manager.cc:1428] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:12:15.519812 13332 ts_tablet_manager.cc:1428] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:12:15.519867 13335 ts_tablet_manager.cc:1397] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:15.520071 13335 raft_consensus.cc:348] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.520188 13335 raft_consensus.cc:374] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.520224 13335 raft_consensus.cc:729] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453488ba38754d108c973672c72f37e2, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.520285 13335 consensus_queue.cc:260] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.520500 13335 ts_tablet_manager.cc:1428] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:12:15.603564 13300 raft_consensus.cc:482] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:15.603688 13300 raft_consensus.cc:504] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.603967 13300 leader_election.cc:288] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:15.604210 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ea420c484d427982c2589ad47149d7" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690" is_pre_election: true
I20260811 10:12:15.604213 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ea420c484d427982c2589ad47149d7" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d" is_pre_election: true
I20260811 10:12:15.604308 13065 raft_consensus.cc:2454] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:15.604342 12816 raft_consensus.cc:2454] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 0.
I20260811 10:12:15.604519 12467 leader_election.cc:302] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:15.604660 13300 raft_consensus.cc:2781] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:15.604717 13300 raft_consensus.cc:482] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:15.604736 13300 raft_consensus.cc:3037] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.605654 13300 raft_consensus.cc:504] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.605795 13300 leader_election.cc:288] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Requested vote from peers b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:15.605988 12816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ea420c484d427982c2589ad47149d7" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b574d070a5484dc79b0b805e85c5834d"
I20260811 10:12:15.605983 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ea420c484d427982c2589ad47149d7" candidate_uuid: "453488ba38754d108c973672c72f37e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690"
I20260811 10:12:15.606073 13065 raft_consensus.cc:3037] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.606076 12816 raft_consensus.cc:3037] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.606768 12816 raft_consensus.cc:2454] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:15.606837 13065 raft_consensus.cc:2454] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 453488ba38754d108c973672c72f37e2 in term 1.
I20260811 10:12:15.606995 12467 leader_election.cc:302] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:15.607106 13300 raft_consensus.cc:2781] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:15.607175 13300 raft_consensus.cc:686] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Becoming Leader. State: Replica: 453488ba38754d108c973672c72f37e2, State: Running, Role: LEADER
I20260811 10:12:15.607234 13300 consensus_queue.cc:237] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.608000 12296 catalog_manager.cc:5495] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 453488ba38754d108c973672c72f37e2 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "453488ba38754d108c973672c72f37e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:15.637629 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.635076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.8.238.1:33003)
0811 10:12:15.635265 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.635267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.635273 (+ 6us) server_negotiation.cc:407] Connection header received
0811 10:12:15.635294 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.635295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.635305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.635384 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:15.635534 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.636063 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.636844 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.637089 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.637171 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.637342 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.637385 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.637448 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.637512 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.637537 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
I20260811 10:12:15.639950 13065 raft_consensus.cc:1261] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:15.639958 12816 raft_consensus.cc:1261] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Refusing update from remote peer 453488ba38754d108c973672c72f37e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:15.640188 13296 consensus_queue.cc:1035] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:15.640336 13300 consensus_queue.cc:1035] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:15.650468 12294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:15.650575 12294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:15.651199 12294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Sending DeleteTablet for 3 replicas of tablet 59ea420c484d427982c2589ad47149d7
I20260811 10:12:15.651453 12545 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ea420c484d427982c2589ad47149d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48654
I20260811 10:12:15.651549 13311 tablet_replica.cc:330] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: stopping tablet replica
I20260811 10:12:15.651554 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ea420c484d427982c2589ad47149d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34328
I20260811 10:12:15.651571 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ea420c484d427982c2589ad47149d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:15 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:60800
I20260811 10:12:15.651598 13311 raft_consensus.cc:2229] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:15.651645 13312 tablet_replica.cc:330] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: stopping tablet replica
I20260811 10:12:15.651652 13311 raft_consensus.cc:2258] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.651700 13312 raft_consensus.cc:2229] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:15.651728 13312 raft_consensus.cc:2258] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.651880 13313 tablet_replica.cc:330] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: stopping tablet replica
I20260811 10:12:15.651949 13313 raft_consensus.cc:2229] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:15.651998 13313 raft_consensus.cc:2258] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:15.652158 13311 ts_tablet_manager.cc:1905] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.652153 12294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:15.652307 12294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:15.652344 13313 ts_tablet_manager.cc:1905] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.652704 13312 ts_tablet_manager.cc:1905] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:15.653934 13311 ts_tablet_manager.cc:1918] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:15.653977 13311 log.cc:1196] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/59ea420c484d427982c2589ad47149d7
I20260811 10:12:15.654049 13312 ts_tablet_manager.cc:1918] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:15.654150 13312 log.cc:1196] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/59ea420c484d427982c2589ad47149d7
I20260811 10:12:15.654155 13311 ts_tablet_manager.cc:1939] T 59ea420c484d427982c2589ad47149d7 P 453488ba38754d108c973672c72f37e2: Deleting consensus metadata
I20260811 10:12:15.654330 13312 ts_tablet_manager.cc:1939] T 59ea420c484d427982c2589ad47149d7 P b574d070a5484dc79b0b805e85c5834d: Deleting consensus metadata
I20260811 10:12:15.654618 12274 catalog_manager.cc:4841] TS b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573): tablet 59ea420c484d427982c2589ad47149d7 (table test-table [id=7cced828741a419badfe09415e4af20f]) successfully deleted
I20260811 10:12:15.654644 12275 catalog_manager.cc:4841] TS 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003): tablet 59ea420c484d427982c2589ad47149d7 (table test-table [id=7cced828741a419badfe09415e4af20f]) successfully deleted
I20260811 10:12:15.654704 13313 ts_tablet_manager.cc:1918] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:15.654752 13313 log.cc:1196] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/59ea420c484d427982c2589ad47149d7
I20260811 10:12:15.654964 13313 ts_tablet_manager.cc:1939] T 59ea420c484d427982c2589ad47149d7 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting consensus metadata
I20260811 10:12:15.655437 12273 catalog_manager.cc:4841] TS 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147): tablet 59ea420c484d427982c2589ad47149d7 (table test-table [id=7cced828741a419badfe09415e4af20f]) successfully deleted
I20260811 10:12:15.655575 12545 tablet_service.cc:1467] Processing CreateTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 (DEFAULT_TABLE table=test-table [id=06f2d259b6fa4f3bb7da5ce56d8465d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.655628 12788 tablet_service.cc:1467] Processing CreateTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 (DEFAULT_TABLE table=test-table [id=06f2d259b6fa4f3bb7da5ce56d8465d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.655719 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339ca1a32f584bc89bc7c216f70a5c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.655782 12788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339ca1a32f584bc89bc7c216f70a5c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.656749 13335 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2: Bootstrap starting.
I20260811 10:12:15.656912 13033 tablet_service.cc:1467] Processing CreateTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 (DEFAULT_TABLE table=test-table [id=06f2d259b6fa4f3bb7da5ce56d8465d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:15.657014 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 339ca1a32f584bc89bc7c216f70a5c09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:15.657107 13335 tablet_bootstrap.cc:654] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.657724 13335 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2: No bootstrap required, opened a new log
I20260811 10:12:15.657793 13335 ts_tablet_manager.cc:1397] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:15.657920 13335 raft_consensus.cc:348] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.657991 13335 raft_consensus.cc:374] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.658025 13335 raft_consensus.cc:729] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453488ba38754d108c973672c72f37e2, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.658078 13335 consensus_queue.cc:260] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.658175 13335 ts_tablet_manager.cc:1428] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:15.658404 13331 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Bootstrap starting.
I20260811 10:12:15.658998 13331 tablet_bootstrap.cc:654] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.659394 13332 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Bootstrap starting.
I20260811 10:12:15.659713 13332 tablet_bootstrap.cc:654] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:15.659771 13331 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: No bootstrap required, opened a new log
I20260811 10:12:15.659830 13331 ts_tablet_manager.cc:1397] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:15.660102 13331 raft_consensus.cc:348] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.660188 13331 raft_consensus.cc:374] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.660214 13331 raft_consensus.cc:729] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.660257 13331 consensus_queue.cc:260] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.660359 13331 ts_tablet_manager.cc:1428] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:15.660676 13332 tablet_bootstrap.cc:492] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: No bootstrap required, opened a new log
I20260811 10:12:15.660729 13332 ts_tablet_manager.cc:1397] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:15.660980 13332 raft_consensus.cc:348] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.661029 13332 raft_consensus.cc:374] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:15.661042 13332 raft_consensus.cc:729] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afcdc80051d4d07b66b7bf4362d5690, State: Initialized, Role: FOLLOWER
I20260811 10:12:15.661077 13332 consensus_queue.cc:260] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.661173 13332 ts_tablet_manager.cc:1428] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:15.672943 13340 raft_consensus.cc:482] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:15.673004 13340 raft_consensus.cc:504] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.673413 13340 leader_election.cc:288] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:15.676987 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.673602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:59407 (local address 127.8.238.1:33003)
0811 10:12:15.673673 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.673675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.673783 (+ 108us) server_negotiation.cc:407] Connection header received
0811 10:12:15.673837 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.673838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.673849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.673917 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:15.674223 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.674953 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676238 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.676615 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676700 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.676713 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.676878 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.676920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:12:15.677011 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.673542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33003 (local address 127.8.238.2:59407)
0811 10:12:15.673690 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:15.673702 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:15.673776 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:15.673942 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:15.673943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:15.673945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:15.674147 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.674149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.675102 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.675103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.676147 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.676150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.676700 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.676786 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.676790 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:15.676866 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:12:15.676915 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":52,"thread_start_us":57,"threads_started":1}
I20260811 10:12:15.677109 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.673626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:49191 (local address 127.8.238.3:43147)
0811 10:12:15.673729 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.673730 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.673792 (+ 62us) server_negotiation.cc:407] Connection header received
0811 10:12:15.673837 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.673838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.673849 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.673926 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:15.674156 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.674960 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676247 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.676691 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676779 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.676792 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.676995 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.677047 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260811 10:12:15.677167 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.673586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.8.238.2:49191)
0811 10:12:15.673725 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:15.673734 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:15.673784 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:15.673963 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:15.673963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:15.673965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:15.674075 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.674079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.675042 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.675043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.676156 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.676160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.676794 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.676795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.676908 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.676911 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:15.676988 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:12:15.677031 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":54,"thread_start_us":86,"threads_started":1}
I20260811 10:12:15.677223 12583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339ca1a32f584bc89bc7c216f70a5c09" candidate_uuid: "b574d070a5484dc79b0b805e85c5834d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "453488ba38754d108c973672c72f37e2" is_pre_election: true
I20260811 10:12:15.677313 12583 raft_consensus.cc:2454] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b574d070a5484dc79b0b805e85c5834d in term 0.
I20260811 10:12:15.677383 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339ca1a32f584bc89bc7c216f70a5c09" candidate_uuid: "b574d070a5484dc79b0b805e85c5834d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690" is_pre_election: true
I20260811 10:12:15.677472 13065 raft_consensus.cc:2454] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b574d070a5484dc79b0b805e85c5834d in term 0.
I20260811 10:12:15.677525 12706 leader_election.cc:302] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:15.677688 13340 raft_consensus.cc:2781] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:15.677721 13340 raft_consensus.cc:482] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:15.677735 13340 raft_consensus.cc:3037] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.678330 13340 raft_consensus.cc:504] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.678476 13340 leader_election.cc:288] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [CANDIDATE]: Term 1 election: Requested vote from peers 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003), 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147)
I20260811 10:12:15.678632 12583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339ca1a32f584bc89bc7c216f70a5c09" candidate_uuid: "b574d070a5484dc79b0b805e85c5834d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "453488ba38754d108c973672c72f37e2"
I20260811 10:12:15.678710 12583 raft_consensus.cc:3037] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.678694 13065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "339ca1a32f584bc89bc7c216f70a5c09" candidate_uuid: "b574d070a5484dc79b0b805e85c5834d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afcdc80051d4d07b66b7bf4362d5690"
I20260811 10:12:15.678791 13065 raft_consensus.cc:3037] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:15.679306 12583 raft_consensus.cc:2454] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b574d070a5484dc79b0b805e85c5834d in term 1.
I20260811 10:12:15.679471 12706 leader_election.cc:302] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 453488ba38754d108c973672c72f37e2, b574d070a5484dc79b0b805e85c5834d; no voters:
I20260811 10:12:15.679483 13065 raft_consensus.cc:2454] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b574d070a5484dc79b0b805e85c5834d in term 1.
I20260811 10:12:15.679633 13340 raft_consensus.cc:2781] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:15.679819 13340 raft_consensus.cc:686] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 1 LEADER]: Becoming Leader. State: Replica: b574d070a5484dc79b0b805e85c5834d, State: Running, Role: LEADER
I20260811 10:12:15.679929 13340 consensus_queue.cc:237] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } }
I20260811 10:12:15.680538 12296 catalog_manager.cc:5495] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d reported cstate change: term changed from 0 to 1, leader changed from <none> to b574d070a5484dc79b0b805e85c5834d (127.8.238.2). New cstate: current_term: 1 leader_uuid: "b574d070a5484dc79b0b805e85c5834d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b574d070a5484dc79b0b805e85c5834d" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 46573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:15.692497 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.689804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.8.238.2:46573)
0811 10:12:15.689983 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.689984 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.689989 (+ 5us) server_negotiation.cc:407] Connection header received
0811 10:12:15.690019 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.690019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.690036 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.690233 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:15.690363 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.690862 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.691597 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.691906 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.691986 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.692138 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.692255 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.692341 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.692383 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.692402 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":132,"threads_started":1}
I20260811 10:12:15.697598 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.695133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:46519 (local address 127.8.238.3:43147)
0811 10:12:15.695170 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.695172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.695386 (+ 214us) server_negotiation.cc:407] Connection header received
0811 10:12:15.695452 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.695452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.695465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.695508 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:15.695644 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.696149 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.696871 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.697077 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.697147 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.697350 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.697388 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.697441 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.697505 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.697527 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:12:15.697633 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.695054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43147 (local address 127.8.238.2:46519)
0811 10:12:15.695315 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:15.695324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:15.695401 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:15.695519 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:15.695520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:15.695522 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:15.695575 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.695576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.696228 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.696230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.696773 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.696775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.697181 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.697182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.697247 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.697260 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.697455 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.697457 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:12:15.697495 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 10:12:15.697539 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":175,"threads_started":1}
I20260811 10:12:15.699817 13065 raft_consensus.cc:1261] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Refusing update from remote peer b574d070a5484dc79b0b805e85c5834d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:15.699898 12583 raft_consensus.cc:1261] T 339ca1a32f584bc89bc7c216f70a5c09 P 453488ba38754d108c973672c72f37e2 [term 1 FOLLOWER]: Refusing update from remote peer b574d070a5484dc79b0b805e85c5834d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:15.700107 13370 consensus_queue.cc:1035] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afcdc80051d4d07b66b7bf4362d5690" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:15.700228 13340 consensus_queue.cc:1035] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [LEADER]: Connected to new peer: Peer: permanent_uuid: "453488ba38754d108c973672c72f37e2" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:15.708149 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.705448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:46573 (local address 127.8.238.3:43093)
0811 10:12:15.705630 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:15.705659 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:15.705877 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:15.706014 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:15.706015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:15.706017 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:15.706061 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.706062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.706715 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.706717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.707423 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:15.707432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.707790 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.707791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:15.707841 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.707846 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.708016 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.708017 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:15.708050 (+ 33us) client_negotiation.cc:240] Negotiation successful
0811 10:12:15.708091 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":132,"threads_started":1}
I20260811 10:12:15.708148 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:15.705484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:43093 (local address 127.8.238.2:46573)
0811 10:12:15.705516 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:15.705518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:15.705862 (+ 344us) server_negotiation.cc:407] Connection header received
0811 10:12:15.705945 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:15.705946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:15.705956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:15.706011 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:15.706118 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.706633 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.707502 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:15.707679 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:15.707731 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:15.707927 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:15.707951 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:15.707989 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:15.708053 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:12:15.708069 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260811 10:12:15.708590 13282 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 10:12:17.231858 12294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:17.231978 12294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:17.232833 12294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b742d1d1bb8c44a38eb338611e240c90: Sending DeleteTablet for 3 replicas of tablet 339ca1a32f584bc89bc7c216f70a5c09
I20260811 10:12:17.233140 12545 tablet_service.cc:1514] Processing DeleteTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:17 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48654
I20260811 10:12:17.233131 12788 tablet_service.cc:1514] Processing DeleteTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:17 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34328
I20260811 10:12:17.233150 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet 339ca1a32f584bc89bc7c216f70a5c09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:17 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:60800
I20260811 10:12:17.233381 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12438
I20260811 10:12:17.243856 13432 tablet_replica.cc:330] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: stopping tablet replica
I20260811 10:12:17.243956 13432 raft_consensus.cc:2229] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:17.244035 13433 tablet_replica.cc:330] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: stopping tablet replica
I20260811 10:12:17.244119 13432 raft_consensus.cc:2258] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:17.244123 13433 raft_consensus.cc:2229] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:17.244172 13433 raft_consensus.cc:2258] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:17.244451 13432 ts_tablet_manager.cc:1905] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:17.244452 13433 ts_tablet_manager.cc:1905] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:17.246204 13432 ts_tablet_manager.cc:1918] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:12:17.246208 13433 ts_tablet_manager.cc:1918] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 10:12:17.246273 13432 log.cc:1196] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/339ca1a32f584bc89bc7c216f70a5c09
I20260811 10:12:17.246273 13433 log.cc:1196] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/339ca1a32f584bc89bc7c216f70a5c09
I20260811 10:12:17.246518 13433 ts_tablet_manager.cc:1939] T 339ca1a32f584bc89bc7c216f70a5c09 P 8afcdc80051d4d07b66b7bf4362d5690: Deleting consensus metadata
I20260811 10:12:17.246522 13432 ts_tablet_manager.cc:1939] T 339ca1a32f584bc89bc7c216f70a5c09 P b574d070a5484dc79b0b805e85c5834d: Deleting consensus metadata
I20260811 10:12:17.246932 12274 catalog_manager.cc:4841] TS b574d070a5484dc79b0b805e85c5834d (127.8.238.2:46573): tablet 339ca1a32f584bc89bc7c216f70a5c09 (table test-table [id=06f2d259b6fa4f3bb7da5ce56d8465d2]) successfully deleted
I20260811 10:12:17.246953 12273 catalog_manager.cc:4841] TS 8afcdc80051d4d07b66b7bf4362d5690 (127.8.238.3:43147): tablet 339ca1a32f584bc89bc7c216f70a5c09 (table test-table [id=06f2d259b6fa4f3bb7da5ce56d8465d2]) successfully deleted
W20260811 10:12:17.248425 12275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:33003 (error 108)
I20260811 10:12:17.248723 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12674
I20260811 10:12:17.260154 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:17.248847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33003 (local address 127.0.0.1:33188)
0811 10:12:17.260016 (+ 11169us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:17.260093 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11133,"thread_start_us":62,"threads_started":1}
W20260811 10:12:17.260334 12275 catalog_manager.cc:4584] TS 453488ba38754d108c973672c72f37e2 (127.8.238.1:33003): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 339ca1a32f584bc89bc7c216f70a5c09: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33003: connect: Connection refused (error 111)
I20260811 10:12:17.264981 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12913
I20260811 10:12:17.275995 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12247
2026-08-11T10:12:17Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5862 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:17 dist-test-slave-7f01 krb5kdc[13440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:17 dist-test-slave-7f01 krb5kdc[13440](info): set up 2 sockets
Aug 11 10:12:17 dist-test-slave-7f01 krb5kdc[13440](info): commencing operation
krb5kdc: starting...
W20260811 10:12:18.616001 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:18 dist-test-slave-7f01 krb5kdc[13440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:18.738999 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39285
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:18.821444 13459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:18.821652 13459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:18.821676 13459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:18.822822 13459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:18.822867 13459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:18.822901 13459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:18.822942 13459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:18.823045 13459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:18.824579 13459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:18.824831 13459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:18.825064 13459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:12:18.830200 13459 server_base.cc:1034] running on GCE node
W20260811 10:12:18.830315 13469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:18.830350 13473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:18.830498 13470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:18.830876 13459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:18.831130 13459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:18.832329 13459 hybrid_clock.cc:648] HybridClock initialized: now 1786443138832308 us; error 31 us; skew 500 ppm
Aug 11 10:12:18 dist-test-slave-7f01 krb5kdc[13440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443138, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:18.834913 13459 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:18.835368 13459 webserver.cc:460] Webserver started at http://127.8.238.62:45153/ using document root <none> and password file <none>
I20260811 10:12:18.835623 13459 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:18.835669 13459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:18.835832 13459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:18.836879 13459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "98f49432e7104673b31ace762b05c64e"
format_stamp: "Formatted at 2026-08-11 10:12:18 on dist-test-slave-7f01"
server_key: "c17e2d93d87256d3b9571ff60d8ef0f5"
server_key_iv: "5ccb5c361109903da3af68d436f9307d"
server_key_version: "encryptionkey@0"
I20260811 10:12:18.837266 13459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "98f49432e7104673b31ace762b05c64e"
format_stamp: "Formatted at 2026-08-11 10:12:18 on dist-test-slave-7f01"
server_key: "c17e2d93d87256d3b9571ff60d8ef0f5"
server_key_iv: "5ccb5c361109903da3af68d436f9307d"
server_key_version: "encryptionkey@0"
I20260811 10:12:18.838840 13459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:18.839685 13491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:18.839919 13459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:18.840021 13459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "98f49432e7104673b31ace762b05c64e"
format_stamp: "Formatted at 2026-08-11 10:12:18 on dist-test-slave-7f01"
server_key: "c17e2d93d87256d3b9571ff60d8ef0f5"
server_key_iv: "5ccb5c361109903da3af68d436f9307d"
server_key_version: "encryptionkey@0"
I20260811 10:12:18.840101 13459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:18.870283 13459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:18.872449 13459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:18.872599 13459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:18.876927 13459 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39285
I20260811 10:12:18.876965 13577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39285 every 8 connection(s)
I20260811 10:12:18.877362 13459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:18.878255 13578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:18.880733 13578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e: Bootstrap starting.
I20260811 10:12:18.881338 13578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:18.881664 13578 log.cc:824] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:18.882398 13578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e: No bootstrap required, opened a new log
I20260811 10:12:18.883940 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13459
I20260811 10:12:18.884020 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:18.884223 9144 external_mini_cluster.cc:1428] Setting key eb5407b9f2587cf9937d35dc27a4dadf
I20260811 10:12:18.884150 13578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } }
I20260811 10:12:18.884318 13578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:18.884389 13578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f49432e7104673b31ace762b05c64e, State: Initialized, Role: FOLLOWER
I20260811 10:12:18.884541 13578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } }
I20260811 10:12:18.884653 13578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:18.884690 13578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:18.884768 13578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:18.885380 13578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } }
I20260811 10:12:18.885499 13578 leader_election.cc:302] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98f49432e7104673b31ace762b05c64e; no voters:
I20260811 10:12:18.885676 13578 leader_election.cc:288] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:18.885728 13585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:18.885946 13585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [term 1 LEADER]: Becoming Leader. State: Replica: 98f49432e7104673b31ace762b05c64e, State: Running, Role: LEADER
I20260811 10:12:18.886081 13585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } }
I20260811 10:12:18.886062 13578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:18.886483 13586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98f49432e7104673b31ace762b05c64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } } }
I20260811 10:12:18.886580 13586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:18.886907 13593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:18.887449 13593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:18.889240 13588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f49432e7104673b31ace762b05c64e. Latest consensus state: current_term: 1 leader_uuid: "98f49432e7104673b31ace762b05c64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f49432e7104673b31ace762b05c64e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39285 } } }
I20260811 10:12:18.889323 13588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:18.889864 13593 catalog_manager.cc:1347] Generated new cluster ID: 47c2cfd839bc4a09afb63501647ac5b4
I20260811 10:12:18.889918 13593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:12:18 dist-test-slave-7f01 krb5kdc[13440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:18.896726 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:18.884726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.8.238.62:39285)
0811 10:12:18.884922 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:18.884933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:18.884988 (+ 55us) server_negotiation.cc:407] Connection header received
0811 10:12:18.885330 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:18.885337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:18.885381 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:18.885531 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:18.886444 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:18.887916 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:18.889078 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:18.889617 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:18.889744 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:18.892534 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:18.892538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:18.892538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:18.892550 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:18.894733 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:18.895431 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:18.895435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:18.895435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:18.895560 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:18.895786 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:18.895787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:18.895788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:18.896133 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:18.896318 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:18.896388 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 10:12:18.896527 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260811 10:12:18.902046 13593 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:18.902539 13593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:18.915541 13593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98f49432e7104673b31ace762b05c64e: Generated new TSK 0
I20260811 10:12:18.915726 13593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:18.947489 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:19.032539 13628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:19.032740 13628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:19.032763 13628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:19.033922 13628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:19.033965 13628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:19.034018 13628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:19.035488 13628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:19.035723 13628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:19.035977 13628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:19.041124 13643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.041095 13635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.041102 13641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:19.041369 13628 server_base.cc:1034] running on GCE node
I20260811 10:12:19.041603 13628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:19.041844 13628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:19.043030 13628 hybrid_clock.cc:648] HybridClock initialized: now 1786443139043007 us; error 34 us; skew 500 ppm
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:19.045621 13628 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:19.046105 13628 webserver.cc:460] Webserver started at http://127.8.238.1:36847/ using document root <none> and password file <none>
I20260811 10:12:19.046329 13628 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:19.046398 13628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:19.046554 13628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:19.047591 13628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "f5a96597e5374cf6910f36fe55b158e2"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "f1a4377e59e4c4a544d0ed9d7747ca78"
server_key_iv: "e3c8fd2004a515c6a7f3b49d32422a43"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.047988 13628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "f5a96597e5374cf6910f36fe55b158e2"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "f1a4377e59e4c4a544d0ed9d7747ca78"
server_key_iv: "e3c8fd2004a515c6a7f3b49d32422a43"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.049474 13628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:19.050241 13652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.050458 13628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:19.050588 13628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "f5a96597e5374cf6910f36fe55b158e2"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "f1a4377e59e4c4a544d0ed9d7747ca78"
server_key_iv: "e3c8fd2004a515c6a7f3b49d32422a43"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.050673 13628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:19.070645 13628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:19.072558 13628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:19.072724 13628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:19.072969 13628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:19.073287 13628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:19.073348 13628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.073381 13628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:19.073417 13628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.079569 13628 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:38425
I20260811 10:12:19.079627 13849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:38425 every 8 connection(s)
I20260811 10:12:19.079938 13628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:19.082577 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13628
I20260811 10:12:19.082665 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:19.082863 9144 external_mini_cluster.cc:1428] Setting key db8e1d5473ceee8f6efac7b75d6de052
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:19.090698 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.080661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39161 (local address 127.8.238.62:39285)
0811 10:12:19.080707 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:19.080709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:19.080934 (+ 225us) server_negotiation.cc:407] Connection header received
0811 10:12:19.081428 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:19.081429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:19.081444 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:19.081499 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:19.082543 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.083192 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.084380 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.084712 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.084792 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.087571 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:19.087572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:19.087573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:19.087581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:19.089104 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.089617 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.089618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.089618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.089669 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.089989 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.089990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.089990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.090118 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:19.090185 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:19.090553 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 10:12:19.090630 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:19.090993 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.080613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39285 (local address 127.8.238.1:39161)
0811 10:12:19.080802 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:19.080843 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:19.081324 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:19.081566 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:19.081570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:19.082182 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:19.082430 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.082437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.083269 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.083271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.084285 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.084288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.084805 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.084806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.084925 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.085472 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:19.085484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:19.087473 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:19.089189 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.089198 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.089200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.089538 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.089732 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.089734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.089734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.089894 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.090204 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:19.090212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:19.090392 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 10:12:19.090522 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 10:12:19.090712 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":98,"threads_started":1}
I20260811 10:12:19.091607 13850 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39285
I20260811 10:12:19.091707 13850 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:19.091928 13850 heartbeater.cc:511] Master 127.8.238.62:39285 requested a full tablet report, sending...
I20260811 10:12:19.092512 13520 ts_manager.cc:194] Registered new tserver with Master: f5a96597e5374cf6910f36fe55b158e2 (127.8.238.1:38425)
I20260811 10:12:19.093158 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:39161
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:19.112425 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:19.196210 13857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:19.196411 13857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:19.196436 13857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:19.197660 13857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:19.197703 13857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:19.197731 13857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:19.199149 13857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:19.199385 13857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:19.199635 13857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:19.204671 13866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.204708 13867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.204713 13870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:19.205121 13857 server_base.cc:1034] running on GCE node
I20260811 10:12:19.205402 13857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:19.205667 13857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:19.206847 13857 hybrid_clock.cc:648] HybridClock initialized: now 1786443139206837 us; error 37 us; skew 500 ppm
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:19.209817 13857 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:19.210291 13857 webserver.cc:460] Webserver started at http://127.8.238.2:39593/ using document root <none> and password file <none>
I20260811 10:12:19.210536 13857 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:19.210613 13857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:19.210789 13857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:19.211992 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "49b1afd6df1a4c8a92285f517966a835"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "cbab559879dd069844ebd8e965b3e2b8"
server_key_iv: "fa768991adedb89023d649e5aee8c4e8"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.212393 13857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "49b1afd6df1a4c8a92285f517966a835"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "cbab559879dd069844ebd8e965b3e2b8"
server_key_iv: "fa768991adedb89023d649e5aee8c4e8"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.213886 13857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:19.214660 13879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.214839 13857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:19.214964 13857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "49b1afd6df1a4c8a92285f517966a835"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "cbab559879dd069844ebd8e965b3e2b8"
server_key_iv: "fa768991adedb89023d649e5aee8c4e8"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.215046 13857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:19.222005 13857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:19.224432 13857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:19.224593 13857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:19.224822 13857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:19.225131 13857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:19.225188 13857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.225222 13857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:19.225250 13857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.231228 13857 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:32945
I20260811 10:12:19.231266 14064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:32945 every 8 connection(s)
I20260811 10:12:19.231581 13857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:19.238075 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13857
I20260811 10:12:19.238180 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:19.238404 9144 external_mini_cluster.cc:1428] Setting key e1817fb253f72cb26ec1f2c34f99c892
I20260811 10:12:19.241200 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.232368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:47803 (local address 127.8.238.62:39285)
0811 10:12:19.232431 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:19.232433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:19.232617 (+ 184us) server_negotiation.cc:407] Connection header received
0811 10:12:19.233097 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:19.233097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:19.233107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:19.233165 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:19.234074 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.234582 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.235569 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.235827 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.235881 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.238104 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:19.238106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:19.238106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:19.238113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:19.239682 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.240237 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.240238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.240239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.240293 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.240580 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.240580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.240580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.240681 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:19.240731 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:19.241063 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 10:12:19.241135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:12:19.241488 14072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.232307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39285 (local address 127.8.238.2:47803)
0811 10:12:19.232485 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:19.232526 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:19.233010 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:19.233232 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:19.233236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:19.233799 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:19.233989 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.233994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.234676 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.234677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.235490 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.235493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.235905 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.235906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.236012 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.236444 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:19.236452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:19.237994 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:19.239786 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.239794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.239797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.240159 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.240362 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.240363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.240364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.240511 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.240744 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:19.240751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:19.240926 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 10:12:19.241043 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:12:19.241225 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260811 10:12:19.241971 14067 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39285
I20260811 10:12:19.242064 14067 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:19.242264 14067 heartbeater.cc:511] Master 127.8.238.62:39285 requested a full tablet report, sending...
I20260811 10:12:19.242664 13520 ts_manager.cc:194] Registered new tserver with Master: 49b1afd6df1a4c8a92285f517966a835 (127.8.238.2:32945)
I20260811 10:12:19.243120 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:47803
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:19.264123 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:19.347435 14075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:19.347632 14075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:19.347654 14075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:19.348816 14075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:19.348860 14075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:19.348886 14075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:19.350329 14075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:19.350569 14075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:19.350793 14075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:19.355917 14086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.355907 14083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:19.355907 14084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:19.356194 14075 server_base.cc:1034] running on GCE node
I20260811 10:12:19.356421 14075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:19.356662 14075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:19.357831 14075 hybrid_clock.cc:648] HybridClock initialized: now 1786443139357830 us; error 40 us; skew 500 ppm
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:19.360507 14075 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:19.360929 14075 webserver.cc:460] Webserver started at http://127.8.238.3:39749/ using document root <none> and password file <none>
I20260811 10:12:19.361176 14075 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:19.361248 14075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:19.361397 14075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:19.362407 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "d6935b319eaf414da2b194db7a31d119"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "d798415ac116f67fc01be37a25375d70"
server_key_iv: "85ffb248b5d252026f79fe92385ea0fd"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.362787 14075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "d6935b319eaf414da2b194db7a31d119"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "d798415ac116f67fc01be37a25375d70"
server_key_iv: "85ffb248b5d252026f79fe92385ea0fd"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.364226 14075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:19.365001 14102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.365338 14075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:19.365454 14075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "d6935b319eaf414da2b194db7a31d119"
format_stamp: "Formatted at 2026-08-11 10:12:19 on dist-test-slave-7f01"
server_key: "d798415ac116f67fc01be37a25375d70"
server_key_iv: "85ffb248b5d252026f79fe92385ea0fd"
server_key_version: "encryptionkey@0"
I20260811 10:12:19.365530 14075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:19.381295 14075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:19.383085 14075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:19.383249 14075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:19.383492 14075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:19.383857 14075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:19.383913 14075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.383945 14075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:19.383972 14075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:19.390647 14075 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33631
I20260811 10:12:19.390698 14302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33631 every 8 connection(s)
I20260811 10:12:19.391002 14075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[13440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443139, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:19.398929 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14075
I20260811 10:12:19.399048 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:19.399267 9144 external_mini_cluster.cc:1428] Setting key fdb26b70eb3cdc55ea31c9500f1d775a
I20260811 10:12:19.399717 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.391637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:42125 (local address 127.8.238.62:39285)
0811 10:12:19.391679 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:19.391681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:19.391925 (+ 244us) server_negotiation.cc:407] Connection header received
0811 10:12:19.392430 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:19.392431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:19.392441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:19.392506 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:19.393421 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.393917 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.394841 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.395023 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.395072 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.397173 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:19.397175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:19.397175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:19.397182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:19.398321 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.398687 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.398688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.398688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.398742 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:19.398992 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:19.398993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:19.398993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:19.399149 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:19.399223 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:19.399536 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 10:12:19.399626 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:12:19.399899 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:19.391599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39285 (local address 127.8.238.3:42125)
0811 10:12:19.391790 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:19.391834 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:19.392344 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:19.392583 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:19.392586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:19.393155 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:19.393342 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.393347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.393997 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.393998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.394770 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:19.394772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:19.395096 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:19.395096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:19.395195 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:19.395626 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:19.395634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:19.397086 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:19.398390 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.398397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.398398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.398620 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.398826 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:19.398827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:19.398827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:19.398924 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:19.399229 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:19.399235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:19.399382 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 10:12:19.399492 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 10:12:19.399645 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260811 10:12:19.400517 14303 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39285
I20260811 10:12:19.400599 14303 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:19.400730 14303 heartbeater.cc:511] Master 127.8.238.62:39285 requested a full tablet report, sending...
I20260811 10:12:19.401078 13520 ts_manager.cc:194] Registered new tserver with Master: d6935b319eaf414da2b194db7a31d119 (127.8.238.3:33631)
I20260811 10:12:19.401461 13520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:42125
I20260811 10:12:19.411181 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 10:12:19.424824 13584 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:19.423722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38470 (local address 127.8.238.62:39285)
0811 10:12:19.423832 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:19.423834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:19.423926 (+ 92us) server_negotiation.cc:407] Connection header received
0811 10:12:19.424041 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:19.424042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:19.424059 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:19.424120 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:19.424751 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38470: BlockingRecv error: recv got EOF from 127.0.0.1:38470 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 10:12:19.425750 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13628
I20260811 10:12:19.444311 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13857
I20260811 10:12:19.452162 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14075
I20260811 10:12:19.463009 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13459
2026-08-11T10:12:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2182 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[14320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[14320](info): set up 2 sockets
Aug 11 10:12:19 dist-test-slave-7f01 krb5kdc[14320](info): commencing operation
krb5kdc: starting...
W20260811 10:12:20.741801 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:20 dist-test-slave-7f01 krb5kdc[14320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:20.866984 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:38457
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:20.954574 14342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:20.954802 14342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:20.954841 14342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:20.956059 14342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:20.956120 14342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:20.956143 14342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:20.956175 14342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:20.956199 14342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:20.957731 14342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:20.958001 14342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:20.958227 14342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:12:20.963101 14342 server_base.cc:1034] running on GCE node
W20260811 10:12:20.963061 14353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:20.963089 14360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:20.963089 14354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:20.963599 14342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:20.963950 14342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:20.965101 14342 hybrid_clock.cc:648] HybridClock initialized: now 1786443140965090 us; error 31 us; skew 500 ppm
Aug 11 10:12:20 dist-test-slave-7f01 krb5kdc[14320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443140, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:20.970809 14342 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:20.971204 14342 webserver.cc:460] Webserver started at http://127.8.238.62:40687/ using document root <none> and password file <none>
I20260811 10:12:20.971392 14342 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:20.971434 14342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:20.971522 14342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:20.972618 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "6586180895b94e7bb5f97811dbf77cee"
format_stamp: "Formatted at 2026-08-11 10:12:20 on dist-test-slave-7f01"
server_key: "624bed8a33cce695c360aa0292841d7c"
server_key_iv: "d51f8161f7a6e41e89c4ace13b614a35"
server_key_version: "encryptionkey@0"
I20260811 10:12:20.972929 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "6586180895b94e7bb5f97811dbf77cee"
format_stamp: "Formatted at 2026-08-11 10:12:20 on dist-test-slave-7f01"
server_key: "624bed8a33cce695c360aa0292841d7c"
server_key_iv: "d51f8161f7a6e41e89c4ace13b614a35"
server_key_version: "encryptionkey@0"
I20260811 10:12:20.974262 14342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:20.974973 14370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:20.975153 14342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:20.975268 14342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "6586180895b94e7bb5f97811dbf77cee"
format_stamp: "Formatted at 2026-08-11 10:12:20 on dist-test-slave-7f01"
server_key: "624bed8a33cce695c360aa0292841d7c"
server_key_iv: "d51f8161f7a6e41e89c4ace13b614a35"
server_key_version: "encryptionkey@0"
I20260811 10:12:20.975345 14342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:21.033187 14342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:21.035717 14342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:21.035953 14342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:21.040218 14342 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:38457
I20260811 10:12:21.040261 14474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:38457 every 8 connection(s)
I20260811 10:12:21.040650 14342 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:21.041579 14477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:21.042768 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14342
I20260811 10:12:21.042887 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:21.043067 9144 external_mini_cluster.cc:1428] Setting key 4861c7a019e6ccbfe94a8028b8ae3756
I20260811 10:12:21.044648 14477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee: Bootstrap starting.
I20260811 10:12:21.045356 14477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:21.045707 14477 log.cc:824] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:21.046554 14477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee: No bootstrap required, opened a new log
I20260811 10:12:21.048481 14477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } }
I20260811 10:12:21.048653 14477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:21.048688 14477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6586180895b94e7bb5f97811dbf77cee, State: Initialized, Role: FOLLOWER
I20260811 10:12:21.048825 14477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } }
I20260811 10:12:21.048925 14477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:21.048972 14477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:21.049027 14477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:21.049544 14477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } }
I20260811 10:12:21.049672 14477 leader_election.cc:302] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6586180895b94e7bb5f97811dbf77cee; no voters:
I20260811 10:12:21.049860 14477 leader_election.cc:288] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:21.049924 14482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:21.050143 14482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [term 1 LEADER]: Becoming Leader. State: Replica: 6586180895b94e7bb5f97811dbf77cee, State: Running, Role: LEADER
I20260811 10:12:21.050159 14477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:21.050307 14482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } }
I20260811 10:12:21.050751 14485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6586180895b94e7bb5f97811dbf77cee. Latest consensus state: current_term: 1 leader_uuid: "6586180895b94e7bb5f97811dbf77cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } } }
I20260811 10:12:21.050884 14485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:21.051134 14496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 10:12:21 dist-test-slave-7f01 krb5kdc[14320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:21.051662 14496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:21.051780 14486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6586180895b94e7bb5f97811dbf77cee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6586180895b94e7bb5f97811dbf77cee" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 38457 } } }
I20260811 10:12:21.052495 14486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:21.054123 14496 catalog_manager.cc:1347] Generated new cluster ID: 69340c89a995494b8a21397c12eb6448
I20260811 10:12:21.054180 14496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:21.055845 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:21.043592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.8.238.62:38457)
0811 10:12:21.043827 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:21.043837 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:21.043872 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:12:21.044426 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:21.044435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:21.044498 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:21.044653 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:21.045328 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:21.046635 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:21.047411 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:21.048768 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:21.048880 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:21.052106 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:21.052111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:21.052111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:21.052122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:21.054196 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:21.054565 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:21.054568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:21.054569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:21.054673 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:21.054898 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:21.054899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:21.054900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:21.055140 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:21.055294 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:21.055417 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:12:21.055560 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":87,"threads_started":1}
I20260811 10:12:21.056839 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:12:21.068225 14496 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:21.068960 14496 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:21.073887 14496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6586180895b94e7bb5f97811dbf77cee: Generated new TSK 0
I20260811 10:12:21.074090 14496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:12:21.078343 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:21.074469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.8.238.62:38457)
0811 10:12:21.074536 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:21.074538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:21.074681 (+ 143us) server_negotiation.cc:407] Connection header received
0811 10:12:21.074734 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:21.074734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:21.074743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:21.074792 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:21.075296 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:21.075807 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:21.076911 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:21.077108 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:21.077162 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:21.077843 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:21.077845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:21.077845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:21.078058 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:21.078075 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:21.078082 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:21.078127 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:21.078227 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 10:12:21.078252 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:12:21.080804 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14342
2026-08-11T10:12:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1630 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:21 dist-test-slave-7f01 krb5kdc[14528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:21 dist-test-slave-7f01 krb5kdc[14528](info): set up 2 sockets
Aug 11 10:12:21 dist-test-slave-7f01 krb5kdc[14528](info): commencing operation
krb5kdc: starting...
W20260811 10:12:22.367193 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:22.488788 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35747
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:22.572417 14558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:22.572616 14558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:22.572638 14558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:22.573830 14558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:22.573885 14558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:22.573900 14558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:22.573912 14558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:22.573925 14558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:22.575416 14558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:22.575676 14558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:22.575953 14558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:22.580801 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:22.580863 14573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:22.580897 14558 server_base.cc:1034] running on GCE node
W20260811 10:12:22.580772 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:22.581243 14558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:22.581593 14558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:22.582765 14558 hybrid_clock.cc:648] HybridClock initialized: now 1786443142582744 us; error 46 us; skew 500 ppm
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:22.585570 14558 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:22.586064 14558 webserver.cc:460] Webserver started at http://127.8.238.62:45987/ using document root <none> and password file <none>
I20260811 10:12:22.586306 14558 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:22.586352 14558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:22.586514 14558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:22.587633 14558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "00905aea501949d8a14bb1884acee468"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "5e11f4ac203de9dea44b5dcb19ffa49b"
server_key_iv: "f26f60c60564b154c376b6459ee78e2c"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.588093 14558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "00905aea501949d8a14bb1884acee468"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "5e11f4ac203de9dea44b5dcb19ffa49b"
server_key_iv: "f26f60c60564b154c376b6459ee78e2c"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.589516 14558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:22.590288 14588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.590495 14558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:22.590615 14558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "00905aea501949d8a14bb1884acee468"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "5e11f4ac203de9dea44b5dcb19ffa49b"
server_key_iv: "f26f60c60564b154c376b6459ee78e2c"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.590694 14558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:22.607551 14558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:22.609927 14558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:22.610100 14558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:22.614573 14558 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35747
I20260811 10:12:22.614635 14692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35747 every 8 connection(s)
I20260811 10:12:22.615032 14558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:22.616225 14693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:22.618647 14693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468: Bootstrap starting.
I20260811 10:12:22.619270 14693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:22.619570 14693 log.cc:824] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:22.620368 14693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468: No bootstrap required, opened a new log
I20260811 10:12:22.622007 14693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } }
I20260811 10:12:22.622152 14693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:22.622175 14693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00905aea501949d8a14bb1884acee468, State: Initialized, Role: FOLLOWER
I20260811 10:12:22.622359 14693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } }
I20260811 10:12:22.622426 14693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:22.622448 14693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:22.622498 14693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:22.623322 14693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } }
I20260811 10:12:22.623442 14693 leader_election.cc:302] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00905aea501949d8a14bb1884acee468; no voters:
I20260811 10:12:22.623625 14693 leader_election.cc:288] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:22.623649 14698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:22.623847 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14558
I20260811 10:12:22.623898 14698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [term 1 LEADER]: Becoming Leader. State: Replica: 00905aea501949d8a14bb1884acee468, State: Running, Role: LEADER
I20260811 10:12:22.623947 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:22.623986 14698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } }
I20260811 10:12:22.623988 14693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:22.624153 9144 external_mini_cluster.cc:1428] Setting key 743bde860a17c3f48e6177e133d58eb1
I20260811 10:12:22.624347 14699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00905aea501949d8a14bb1884acee468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } } }
I20260811 10:12:22.624447 14699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:22.624661 14700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00905aea501949d8a14bb1884acee468. Latest consensus state: current_term: 1 leader_uuid: "00905aea501949d8a14bb1884acee468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00905aea501949d8a14bb1884acee468" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35747 } } }
I20260811 10:12:22.624794 14700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:22.624975 14704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:22.625582 14704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:22.627276 14704 catalog_manager.cc:1347] Generated new cluster ID: 71b3f2b0e117479298f3161a095cdf1c
I20260811 10:12:22.627336 14704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:22.635551 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:22.624789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.8.238.62:35747)
0811 10:12:22.625041 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:22.625054 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:22.625090 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:12:22.625225 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:22.625234 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:22.625290 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:22.626203 (+ 913us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:22.626500 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.627731 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.628520 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.628959 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.629090 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:22.631567 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:22.631574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:22.631574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:22.631588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:22.633884 (+ 2296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.634284 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.634287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.634287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.634398 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.634618 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.634619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.634620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.634868 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:22.635055 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:22.635124 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 10:12:22.635375 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":105,"threads_started":1}
I20260811 10:12:22.636952 14704 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:22.637440 14704 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:22.642331 14704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00905aea501949d8a14bb1884acee468: Generated new TSK 0
I20260811 10:12:22.642563 14704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:22.678154 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:22.775595 14741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:22.775852 14741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:22.775894 14741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:22.777078 14741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:22.777163 14741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:22.777207 14741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:22.778733 14741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:22.779026 14741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:22.779311 14741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:22.784605 14750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:22.784638 14741 server_base.cc:1034] running on GCE node
W20260811 10:12:22.784654 14751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:22.784605 14753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:22.785015 14741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:22.785244 14741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:22.786365 14741 hybrid_clock.cc:648] HybridClock initialized: now 1786443142786353 us; error 29 us; skew 500 ppm
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:22.789058 14741 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:22.789502 14741 webserver.cc:460] Webserver started at http://127.8.238.1:40367/ using document root <none> and password file <none>
I20260811 10:12:22.789731 14741 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:22.789801 14741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:22.789960 14741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:22.791023 14741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "69dacf24146f46e4971aa7ac15e41ba9"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "77c3599efa033018d21578a33244af1e"
server_key_iv: "43797be00bca10185a74ada44cd7716a"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.791407 14741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "69dacf24146f46e4971aa7ac15e41ba9"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "77c3599efa033018d21578a33244af1e"
server_key_iv: "43797be00bca10185a74ada44cd7716a"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.792867 14741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:22.793699 14765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.793887 14741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:22.794003 14741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "69dacf24146f46e4971aa7ac15e41ba9"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "77c3599efa033018d21578a33244af1e"
server_key_iv: "43797be00bca10185a74ada44cd7716a"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.794080 14741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:22.809043 14741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:22.810846 14741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:22.811017 14741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:22.811244 14741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:22.811571 14741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:22.811630 14741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.811666 14741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:22.811694 14741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.818029 14741 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34297
I20260811 10:12:22.818084 14969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34297 every 8 connection(s)
I20260811 10:12:22.818392 14741 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:22.824008 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14741
I20260811 10:12:22.824119 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:22.824318 9144 external_mini_cluster.cc:1428] Setting key 5de973b4d0291a32f83f5289186e8534
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:22.828058 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:22.819091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:34841 (local address 127.8.238.62:35747)
0811 10:12:22.819167 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:22.819169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:22.819374 (+ 205us) server_negotiation.cc:407] Connection header received
0811 10:12:22.819864 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:22.819865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:22.819876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:22.819927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:22.820813 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.821310 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.822267 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.822464 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.822514 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:22.824970 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:22.824971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:22.824972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:22.824980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:22.826485 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.827010 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.827012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.827013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.827079 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.827353 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.827355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.827355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.827477 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:22.827556 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:22.827873 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 10:12:22.827967 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:12:22.828233 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:22.819070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35747 (local address 127.8.238.1:34841)
0811 10:12:22.819243 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:22.819278 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:22.819781 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:22.819977 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:22.819980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:22.820545 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:22.820738 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:22.820743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.821395 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.821396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:22.822196 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:22.822198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.822538 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.822539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:22.822642 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:22.823060 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:22.823068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:22.824867 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:22.826568 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:22.826576 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:22.826578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:22.826940 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:22.827159 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:22.827160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:22.827160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:22.827282 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:22.827548 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:22.827555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:22.827695 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 10:12:22.827836 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 10:12:22.827997 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260811 10:12:22.828791 14970 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35747
I20260811 10:12:22.828899 14970 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:22.829095 14970 heartbeater.cc:511] Master 127.8.238.62:35747 requested a full tablet report, sending...
I20260811 10:12:22.829602 14625 ts_manager.cc:194] Registered new tserver with Master: 69dacf24146f46e4971aa7ac15e41ba9 (127.8.238.1:34297)
I20260811 10:12:22.830231 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:34841
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:22.851038 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:22.935074 14978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:22.935269 14978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:22.935293 14978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:22.936501 14978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:22.936548 14978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:22.936574 14978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:22.937934 14978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:22.938122 14978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:22.938288 14978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:22.943308 14984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:22.943339 14985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:22.943397 14988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:22.943640 14978 server_base.cc:1034] running on GCE node
I20260811 10:12:22.943864 14978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:22.944101 14978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:22.945245 14978 hybrid_clock.cc:648] HybridClock initialized: now 1786443142945235 us; error 34 us; skew 500 ppm
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:22.947916 14978 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:22.948335 14978 webserver.cc:460] Webserver started at http://127.8.238.2:34293/ using document root <none> and password file <none>
I20260811 10:12:22.948585 14978 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:22.948654 14978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:22.948801 14978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:22.949826 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "616e590b5ae24ff3b0faf80e067c679b"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "39050fcf7c7bef9a941577c6e1cd3e93"
server_key_iv: "044138411debf82da204ea1ba8e21c54"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.950209 14978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "616e590b5ae24ff3b0faf80e067c679b"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "39050fcf7c7bef9a941577c6e1cd3e93"
server_key_iv: "044138411debf82da204ea1ba8e21c54"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.951534 14978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:12:22.952306 14999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.952466 14978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.952567 14978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "616e590b5ae24ff3b0faf80e067c679b"
format_stamp: "Formatted at 2026-08-11 10:12:22 on dist-test-slave-7f01"
server_key: "39050fcf7c7bef9a941577c6e1cd3e93"
server_key_iv: "044138411debf82da204ea1ba8e21c54"
server_key_version: "encryptionkey@0"
I20260811 10:12:22.952648 14978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:22.979324 14978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:22.981925 14978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:22.982101 14978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:22.982360 14978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:22.982690 14978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:22.982746 14978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.982792 14978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:22.982822 14978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:22.989339 14978 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:45527
I20260811 10:12:22.989380 15182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:45527 every 8 connection(s)
I20260811 10:12:22.989739 14978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:22 dist-test-slave-7f01 krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:22.996291 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14978
I20260811 10:12:22.996392 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:22.996613 9144 external_mini_cluster.cc:1428] Setting key 132f25e55651c5b0be3f5deccbe714b9
I20260811 10:12:23.000102 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:22.990472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:34923 (local address 127.8.238.62:35747)
0811 10:12:22.990567 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:22.990569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:22.990757 (+ 188us) server_negotiation.cc:407] Connection header received
0811 10:12:22.991252 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:22.991253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:22.991265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:22.991318 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:22.992357 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.992862 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.993961 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.994195 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.994248 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:22.996723 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:22.996725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:22.996725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:22.996733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:22.998434 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.998946 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.998947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.998948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.999018 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:22.999315 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:22.999316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:22.999316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:22.999489 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:22.999562 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:22.999881 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 10:12:22.999977 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 10:12:23.000141 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:22.990442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35747 (local address 127.8.238.2:34923)
0811 10:12:22.990636 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:22.990673 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:22.991164 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:22.991371 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:22.991375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:22.992065 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:22.992276 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:22.992281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.992991 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.992992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:22.993888 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:22.993890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:22.994273 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:22.994273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:22.994381 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:22.994996 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:22.995004 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:22.996603 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:22.998521 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:22.998529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:22.998531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:22.998859 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:22.999089 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:22.999090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:22.999090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:22.999241 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:22.999567 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:22.999574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:22.999703 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 10:12:22.999841 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 10:12:22.999984 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260811 10:12:23.000679 15184 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35747
I20260811 10:12:23.000778 15184 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:23.000974 15184 heartbeater.cc:511] Master 127.8.238.62:35747 requested a full tablet report, sending...
I20260811 10:12:23.001420 14625 ts_manager.cc:194] Registered new tserver with Master: 616e590b5ae24ff3b0faf80e067c679b (127.8.238.2:45527)
I20260811 10:12:23.001937 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:34923
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:23.024838 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:23.110345 15197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:23.110543 15197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:23.110565 15197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:23.111816 15197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:23.111860 15197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:23.111887 15197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:23.113261 15197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:23.113468 15197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:23.113646 15197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:23.118681 15205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:23.118732 15206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:23.118743 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:23.119064 15197 server_base.cc:1034] running on GCE node
I20260811 10:12:23.119261 15197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:23.119521 15197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:23.120697 15197 hybrid_clock.cc:648] HybridClock initialized: now 1786443143120683 us; error 30 us; skew 500 ppm
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:23.123380 15197 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:23.123935 15197 webserver.cc:460] Webserver started at http://127.8.238.3:38821/ using document root <none> and password file <none>
I20260811 10:12:23.124202 15197 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:23.124274 15197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:23.124423 15197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:23.125590 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "d08d3bb78e64495eaf3fab3de649f765"
format_stamp: "Formatted at 2026-08-11 10:12:23 on dist-test-slave-7f01"
server_key: "3ddbc9ecb6e09d04397393c17b1d09d5"
server_key_iv: "baf8a80313648a0d9df8a61294216b27"
server_key_version: "encryptionkey@0"
I20260811 10:12:23.125986 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "d08d3bb78e64495eaf3fab3de649f765"
format_stamp: "Formatted at 2026-08-11 10:12:23 on dist-test-slave-7f01"
server_key: "3ddbc9ecb6e09d04397393c17b1d09d5"
server_key_iv: "baf8a80313648a0d9df8a61294216b27"
server_key_version: "encryptionkey@0"
I20260811 10:12:23.127358 15197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:23.128139 15220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:23.128325 15197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:23.128438 15197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "d08d3bb78e64495eaf3fab3de649f765"
format_stamp: "Formatted at 2026-08-11 10:12:23 on dist-test-slave-7f01"
server_key: "3ddbc9ecb6e09d04397393c17b1d09d5"
server_key_iv: "baf8a80313648a0d9df8a61294216b27"
server_key_version: "encryptionkey@0"
I20260811 10:12:23.128513 15197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:23.160225 15197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:23.162546 15197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:23.162706 15197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:23.162950 15197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:23.163270 15197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:23.163339 15197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:23.163383 15197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:23.163409 15197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:23.169291 15197 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43073
I20260811 10:12:23.169334 15405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43073 every 8 connection(s)
I20260811 10:12:23.169633 15197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:23.170394 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15197
I20260811 10:12:23.170501 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:23.170709 9144 external_mini_cluster.cc:1428] Setting key 17f1e3c69ccab72e1359b9eb513723ff
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:23.178824 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:23.170359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:41089 (local address 127.8.238.62:35747)
0811 10:12:23.170407 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:23.170409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:23.170693 (+ 284us) server_negotiation.cc:407] Connection header received
0811 10:12:23.171393 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:23.171394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:23.171408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:23.171458 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:23.172466 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.172963 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.173966 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.174162 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.174238 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:23.176248 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:23.176250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:23.176250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:23.176258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:23.177456 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:23.177853 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:23.177854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:23.177855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:23.177918 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:23.178174 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:23.178175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:23.178175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:23.178296 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:23.178370 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:23.178643 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 10:12:23.178733 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:12:23.179005 15411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:23.170351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35747 (local address 127.8.238.3:41089)
0811 10:12:23.170552 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:23.170599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:23.171281 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:23.171524 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:23.171528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:23.172177 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:23.172367 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:23.172372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.173060 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.173062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:23.173873 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:23.173876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.174247 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.174248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:23.174382 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:23.174826 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:23.174833 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:23.176150 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:23.177540 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:23.177547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:23.177550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:23.177767 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:23.177994 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:23.177994 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:23.177995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:23.178098 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:23.178372 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:23.178380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:23.178512 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 10:12:23.178619 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 10:12:23.178768 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260811 10:12:23.179456 15406 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35747
I20260811 10:12:23.179551 15406 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:23.179687 15406 heartbeater.cc:511] Master 127.8.238.62:35747 requested a full tablet report, sending...
I20260811 10:12:23.180142 14625 ts_manager.cc:194] Registered new tserver with Master: d08d3bb78e64495eaf3fab3de649f765 (127.8.238.3:43073)
I20260811 10:12:23.180485 14625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:41089
I20260811 10:12:23.182451 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[14528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443143, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[14528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443143, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:23.206573 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:23.199359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.8.238.62:35747)
0811 10:12:23.199445 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:23.199447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:23.199542 (+ 95us) server_negotiation.cc:407] Connection header received
0811 10:12:23.199629 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:23.199630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:23.199640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:23.199704 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:23.200357 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.200844 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.201588 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:23.201783 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:23.201855 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:23.204122 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:23.204124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:23.204125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:23.204132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:23.205380 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:23.205732 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:23.205734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:23.205734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:23.205799 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:23.206012 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:23.206013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:23.206013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:23.206167 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:23.206286 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:23.206377 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 10:12:23.206459 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 10:12:23.206828 14625 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51370
I20260811 10:12:23.207405 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14741
I20260811 10:12:23.218755 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14978
I20260811 10:12:23.241040 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15197
I20260811 10:12:23.250496 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14558
2026-08-11T10:12:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2161 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[15425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[15425](info): set up 2 sockets
Aug 11 10:12:23 dist-test-slave-7f01 krb5kdc[15425](info): commencing operation
krb5kdc: starting...
W20260811 10:12:24.521530 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:24 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:24.643142 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:36909
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:36909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:24.726310 15442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:24.726506 15442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:24.726529 15442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:24.727628 15442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:24.727667 15442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:24.727681 15442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:24.727694 15442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:24.727706 15442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:24.729137 15442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:36909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:24.729318 15442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:24.729480 15442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:24.734273 15454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:24.734261 15449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:24.734302 15450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:24.734709 15442 server_base.cc:1034] running on GCE node
I20260811 10:12:24.734997 15442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:24.735275 15442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:24.736446 15442 hybrid_clock.cc:648] HybridClock initialized: now 1786443144736420 us; error 36 us; skew 500 ppm
Aug 11 10:12:24 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:24.738932 15442 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:24.739425 15442 webserver.cc:460] Webserver started at http://127.8.238.62:44611/ using document root <none> and password file <none>
I20260811 10:12:24.739676 15442 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:24.739768 15442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:24.739926 15442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:24.740964 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ae9634e43df941b1b77a1ce7f25271a2"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "8424424c12e3af1238abf11340a89446"
server_key_iv: "fee2207443d4e815791f3bf778b31a0b"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.741336 15442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ae9634e43df941b1b77a1ce7f25271a2"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "8424424c12e3af1238abf11340a89446"
server_key_iv: "fee2207443d4e815791f3bf778b31a0b"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.742739 15442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:24.743487 15471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:24.743681 15442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:24.743824 15442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ae9634e43df941b1b77a1ce7f25271a2"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "8424424c12e3af1238abf11340a89446"
server_key_iv: "fee2207443d4e815791f3bf778b31a0b"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.743904 15442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:24.784281 15442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:24.786363 15442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:24.786545 15442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:24.791049 15442 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:36909
I20260811 10:12:24.791088 15577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:36909 every 8 connection(s)
I20260811 10:12:24.791639 15442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:24.792415 15578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:24.794797 15578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2: Bootstrap starting.
I20260811 10:12:24.795437 15578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:24.795779 15578 log.cc:824] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:24.796576 15578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2: No bootstrap required, opened a new log
I20260811 10:12:24.798223 15578 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } }
I20260811 10:12:24.798447 15578 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:24.798478 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15442
I20260811 10:12:24.798493 15578 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae9634e43df941b1b77a1ce7f25271a2, State: Initialized, Role: FOLLOWER
I20260811 10:12:24.798592 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:24.798650 15578 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } }
I20260811 10:12:24.798730 15578 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:24.798772 15578 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:24.798797 9144 external_mini_cluster.cc:1428] Setting key ae0e686638c985381281db396a82be6c
I20260811 10:12:24.798825 15578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:24.800079 15578 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } }
I20260811 10:12:24.800223 15578 leader_election.cc:302] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae9634e43df941b1b77a1ce7f25271a2; no voters:
I20260811 10:12:24.800447 15578 leader_election.cc:288] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:24.800514 15586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:24.800680 15586 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [term 1 LEADER]: Becoming Leader. State: Replica: ae9634e43df941b1b77a1ce7f25271a2, State: Running, Role: LEADER
I20260811 10:12:24.800725 15578 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:24.800812 15586 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } }
I20260811 10:12:24.801210 15586 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae9634e43df941b1b77a1ce7f25271a2. Latest consensus state: current_term: 1 leader_uuid: "ae9634e43df941b1b77a1ce7f25271a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } } }
I20260811 10:12:24.801219 15587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae9634e43df941b1b77a1ce7f25271a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae9634e43df941b1b77a1ce7f25271a2" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36909 } } }
I20260811 10:12:24.801385 15586 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:24.801393 15587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:24.802204 15599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:24.802796 15599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:24.804450 15599 catalog_manager.cc:1347] Generated new cluster ID: 26333a78d473404096663fd432c0ab74
I20260811 10:12:24.804517 15599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:12:24 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:24.810418 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:24.799295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.8.238.62:36909)
0811 10:12:24.799491 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:24.799502 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:24.799532 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:12:24.799623 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:24.799629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:24.799669 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:24.799848 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:24.800703 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:24.802217 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:24.803143 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:24.803580 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:24.803719 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:24.806504 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:24.806508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:24.806508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:24.806521 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:24.808559 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:24.808950 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:24.808960 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:24.808960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:24.809095 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:24.809304 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:24.809305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:24.809305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:24.809613 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:24.809811 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:24.809899 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 10:12:24.810223 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":117,"threads_started":1}
I20260811 10:12:24.816815 15599 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:24.817305 15599 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:24.833720 15599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae9634e43df941b1b77a1ce7f25271a2: Generated new TSK 0
I20260811 10:12:24.833920 15599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:24.870220 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:24.961177 15623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:24.961400 15623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:24.961439 15623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:24.962576 15623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:24.962641 15623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:24.962685 15623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:24.964234 15623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:24.964496 15623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:24.964721 15623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:24.969872 15634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:24.969996 15623 server_base.cc:1034] running on GCE node
W20260811 10:12:24.970070 15632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:24.969885 15631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:24.970323 15623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:24.970613 15623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:24.971796 15623 hybrid_clock.cc:648] HybridClock initialized: now 1786443144971765 us; error 42 us; skew 500 ppm
Aug 11 10:12:24 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:24.974449 15623 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:24.974905 15623 webserver.cc:460] Webserver started at http://127.8.238.1:34611/ using document root <none> and password file <none>
I20260811 10:12:24.975108 15623 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:24.975147 15623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:24.975234 15623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:24.976289 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "9efdd353dc984117bdeafdc4c032d735"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "0d70939c17405564300386a515cf36cb"
server_key_iv: "40c4cd99afeeaf6a49cd130b4c8531e0"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.976600 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "9efdd353dc984117bdeafdc4c032d735"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "0d70939c17405564300386a515cf36cb"
server_key_iv: "40c4cd99afeeaf6a49cd130b4c8531e0"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.977953 15623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:24.978705 15648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:24.978947 15623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:24.979043 15623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "9efdd353dc984117bdeafdc4c032d735"
format_stamp: "Formatted at 2026-08-11 10:12:24 on dist-test-slave-7f01"
server_key: "0d70939c17405564300386a515cf36cb"
server_key_iv: "40c4cd99afeeaf6a49cd130b4c8531e0"
server_key_version: "encryptionkey@0"
I20260811 10:12:24.979141 15623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:24.998602 15623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:25.001044 15623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:25.001202 15623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:25.001437 15623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:25.001886 15623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:25.001931 15623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.001955 15623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:25.001997 15623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.008491 15623 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:42563
I20260811 10:12:25.008549 15833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:42563 every 8 connection(s)
I20260811 10:12:25.008857 15623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:25.016175 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15623
I20260811 10:12:25.016412 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:25.016621 9144 external_mini_cluster.cc:1428] Setting key 275ab9b63d6a7f4e1a29ac8f3fe51ce1
I20260811 10:12:25.018498 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.009563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:48471 (local address 127.8.238.62:36909)
0811 10:12:25.009609 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:25.009611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:25.009833 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:12:25.010752 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:25.010753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:25.010764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:25.010814 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:25.011412 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.011961 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.012926 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.013115 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.013166 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.015481 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:25.015482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:25.015482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:25.015488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:25.017066 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.017493 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.017495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.017495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.017557 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.017816 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.017817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.017817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.017936 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:25.018010 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:25.018293 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 10:12:25.018393 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:25.018656 15840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.009552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:36909 (local address 127.8.238.1:48471)
0811 10:12:25.009710 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:25.009745 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:25.010662 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:25.010875 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:25.010878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:25.011136 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:25.011327 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.011332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.012041 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.012042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.012850 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.012852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.013212 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.013213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.013340 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.013925 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:25.013934 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:25.015395 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:25.017159 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.017166 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.017169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.017403 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.017623 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.017624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.017625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.017739 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.018015 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:25.018023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:25.018161 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 10:12:25.018267 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 10:12:25.018423 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260811 10:12:25.019219 15834 heartbeater.cc:347] Connected to a master server at 127.8.238.62:36909
I20260811 10:12:25.019327 15834 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:25.019531 15834 heartbeater.cc:511] Master 127.8.238.62:36909 requested a full tablet report, sending...
I20260811 10:12:25.020098 15502 ts_manager.cc:194] Registered new tserver with Master: 9efdd353dc984117bdeafdc4c032d735 (127.8.238.1:42563)
I20260811 10:12:25.020788 15502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:48471
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:25.043121 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:25.128376 15844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:25.128589 15844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:25.128612 15844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:25.129786 15844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:25.129827 15844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:25.129859 15844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:25.131320 15844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:25.131508 15844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:25.131678 15844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:25.136857 15854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:25.136861 15860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:25.136929 15856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:25.136989 15844 server_base.cc:1034] running on GCE node
I20260811 10:12:25.137286 15844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:25.137501 15844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:25.138667 15844 hybrid_clock.cc:648] HybridClock initialized: now 1786443145138653 us; error 50 us; skew 500 ppm
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:25.141207 15844 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:25.141700 15844 webserver.cc:460] Webserver started at http://127.8.238.2:36043/ using document root <none> and password file <none>
I20260811 10:12:25.141975 15844 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:25.142045 15844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:25.142216 15844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:25.143260 15844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "820cc81ce5064c039973fa90250c7747"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "e35568b5495618f777e10ed6e92af5dc"
server_key_iv: "fd24980e01501b1669fb1b0a59b2619d"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.143635 15844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "820cc81ce5064c039973fa90250c7747"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "e35568b5495618f777e10ed6e92af5dc"
server_key_iv: "fd24980e01501b1669fb1b0a59b2619d"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.145020 15844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:25.145750 15870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.145970 15844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:25.146085 15844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "820cc81ce5064c039973fa90250c7747"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "e35568b5495618f777e10ed6e92af5dc"
server_key_iv: "fd24980e01501b1669fb1b0a59b2619d"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.146158 15844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:25.153528 15844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:25.155526 15844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:25.155681 15844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:25.155977 15844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:25.156297 15844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:25.156355 15844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.156395 15844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:25.156423 15844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.162336 15844 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:37423
I20260811 10:12:25.162416 16074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:37423 every 8 connection(s)
I20260811 10:12:25.162716 15844 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:25.168856 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15844
I20260811 10:12:25.168967 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:25.169160 9144 external_mini_cluster.cc:1428] Setting key c97f429f637c32dd5dcb24fcc300dff6
I20260811 10:12:25.172168 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.163427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:48181 (local address 127.8.238.62:36909)
0811 10:12:25.163493 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:25.163495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:25.163684 (+ 189us) server_negotiation.cc:407] Connection header received
0811 10:12:25.164629 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:25.164630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:25.164640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:25.164696 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:25.165253 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.165764 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.166695 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.166928 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.166984 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.169047 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:25.169048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:25.169049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:25.169057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:25.170563 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.171051 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.171052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.171053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.171109 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.171403 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.171403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.171404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.171504 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:25.171554 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:25.171940 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 10:12:25.172043 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:12:25.172187 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.163391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:36909 (local address 127.8.238.2:48181)
0811 10:12:25.163557 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:25.163596 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:25.164538 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:25.164742 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:25.164745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:25.164991 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:25.165171 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.165175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.165844 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.165845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.166619 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.166621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.167010 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.167011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.167112 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.167510 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:25.167518 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:25.168943 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:25.170647 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.170655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.170657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.170974 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.171174 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.171174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.171175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.171329 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.171568 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:25.171575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:25.171766 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 10:12:25.171890 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:12:25.172040 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260811 10:12:25.172670 16075 heartbeater.cc:347] Connected to a master server at 127.8.238.62:36909
I20260811 10:12:25.172760 16075 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:25.172958 16075 heartbeater.cc:511] Master 127.8.238.62:36909 requested a full tablet report, sending...
I20260811 10:12:25.173365 15502 ts_manager.cc:194] Registered new tserver with Master: 820cc81ce5064c039973fa90250c7747 (127.8.238.2:37423)
I20260811 10:12:25.173863 15502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:48181
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:25.196563 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:25.278635 16083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:25.278879 16083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:25.278916 16083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:25.280076 16083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:25.280149 16083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:25.280195 16083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:25.281703 16083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:36909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:25.281970 16083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:25.282244 16083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:25.287149 16095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:25.287202 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:25.287323 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:25.287564 16083 server_base.cc:1034] running on GCE node
I20260811 10:12:25.287824 16083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:25.288066 16083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:25.289211 16083 hybrid_clock.cc:648] HybridClock initialized: now 1786443145289192 us; error 38 us; skew 500 ppm
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:25.291965 16083 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:25.292450 16083 webserver.cc:460] Webserver started at http://127.8.238.3:42813/ using document root <none> and password file <none>
I20260811 10:12:25.292696 16083 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:25.292766 16083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:25.292928 16083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:25.293955 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "3cf75f2969f54d4584a19b597775c2c2"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "70cd8ad45d934455b5fe23116919cca6"
server_key_iv: "5cb05e77b9c11c516a2a705afdccb3a7"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.294325 16083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "3cf75f2969f54d4584a19b597775c2c2"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "70cd8ad45d934455b5fe23116919cca6"
server_key_iv: "5cb05e77b9c11c516a2a705afdccb3a7"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.295665 16083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:12:25.296478 16106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.296657 16083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:25.296761 16083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "3cf75f2969f54d4584a19b597775c2c2"
format_stamp: "Formatted at 2026-08-11 10:12:25 on dist-test-slave-7f01"
server_key: "70cd8ad45d934455b5fe23116919cca6"
server_key_iv: "5cb05e77b9c11c516a2a705afdccb3a7"
server_key_version: "encryptionkey@0"
I20260811 10:12:25.296849 16083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:25.315433 16083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:25.317790 16083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:25.317970 16083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:25.318209 16083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:25.318521 16083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:25.318575 16083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.318616 16083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:25.318643 16083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:25.324848 16083 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:42389
I20260811 10:12:25.324901 16280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:42389 every 8 connection(s)
I20260811 10:12:25.325217 16083 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:25.331275 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16083
I20260811 10:12:25.331353 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:25.331564 9144 external_mini_cluster.cc:1428] Setting key 5ae7a0fe77b96e7f9fd4093b4333e68c
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:25.335443 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.325945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:52099 (local address 127.8.238.62:36909)
0811 10:12:25.326012 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:25.326014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:25.326231 (+ 217us) server_negotiation.cc:407] Connection header received
0811 10:12:25.327244 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:25.327245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:25.327260 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:25.327307 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:25.328024 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.328539 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.329553 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.329758 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.329861 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.332218 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:25.332220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:25.332221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:25.332229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:25.333954 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.334444 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.334445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.334446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.334514 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.334779 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.334780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.334780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.334909 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:25.334988 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:25.335265 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 10:12:25.335346 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:12:25.335640 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.325919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:36909 (local address 127.8.238.3:52099)
0811 10:12:25.326114 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:25.326150 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:25.327120 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:25.327390 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:25.327394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:25.327659 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:25.327928 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.327934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.328636 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.328637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.329459 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:25.329460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.329873 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.329873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:25.330014 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.330467 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:25.330475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:25.332115 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:25.334049 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.334057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.334059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.334362 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.334583 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:25.334584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:25.334584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:25.334701 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:25.334977 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:25.334985 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:25.335129 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 10:12:25.335240 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 10:12:25.335408 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260811 10:12:25.336094 16286 heartbeater.cc:347] Connected to a master server at 127.8.238.62:36909
I20260811 10:12:25.336177 16286 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:25.336386 16286 heartbeater.cc:511] Master 127.8.238.62:36909 requested a full tablet report, sending...
I20260811 10:12:25.336740 15502 ts_manager.cc:194] Registered new tserver with Master: 3cf75f2969f54d4584a19b597775c2c2 (127.8.238.3:42389)
I20260811 10:12:25.337218 15502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:52099
I20260811 10:12:25.343187 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:25.367612 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.359967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.8.238.1:42563)
0811 10:12:25.360152 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:25.360157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:25.360168 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:25.360196 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:25.360198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:25.360211 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:25.360278 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:25.360847 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.361564 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.362293 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.362604 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.362659 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.364871 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:25.364880 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:25.364881 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:25.364887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:25.366511 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.366866 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.366868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.366869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.366938 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.367129 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.367129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.367130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.367260 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:25.367372 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:25.367468 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 10:12:25.367532 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[15425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:25.375675 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:25.369093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43614 (local address 127.8.238.62:36909)
0811 10:12:25.369161 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:25.369163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:25.369172 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:25.369228 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:25.369229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:25.369238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:25.369284 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:25.369722 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.370192 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.370804 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:25.370987 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:25.371036 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:25.373237 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:25.373239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:25.373239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:25.373245 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:25.374622 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.374935 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.374936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.374936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.374997 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:25.375166 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:25.375166 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:25.375167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:25.375305 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:25.375380 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:25.375503 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:12:25.375563 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 10:12:25.375975 15502 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43614
I20260811 10:12:25.376379 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15623
I20260811 10:12:25.389880 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15844
I20260811 10:12:25.417358 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16083
I20260811 10:12:25.428686 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15442
2026-08-11T10:12:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2175 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[16303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[16303](info): set up 2 sockets
Aug 11 10:12:25 dist-test-slave-7f01 krb5kdc[16303](info): commencing operation
krb5kdc: starting...
W20260811 10:12:26.705681 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:26 dist-test-slave-7f01 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:26.827280 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40393
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:26.910282 16331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:26.910512 16331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:26.910552 16331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:26.911703 16331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:26.911783 16331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:26.911810 16331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:26.911842 16331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:26.911865 16331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:26.913362 16331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:26.913635 16331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:26.913856 16331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:26.918656 16340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:26.918675 16336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:26.918672 16337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:26.918938 16331 server_base.cc:1034] running on GCE node
I20260811 10:12:26.919222 16331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:26.919540 16331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:26.920703 16331 hybrid_clock.cc:648] HybridClock initialized: now 1786443146920691 us; error 42 us; skew 500 ppm
Aug 11 10:12:26 dist-test-slave-7f01 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443146, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:26.927695 16331 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:26.928161 16331 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:26.928238 16331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:26.928399 16331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:26.929482 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba"
format_stamp: "Formatted at 2026-08-11 10:12:26 on dist-test-slave-7f01"
server_key: "0ae7b119680b9f6a17c3f70c6bbc6772"
server_key_iv: "e12657e662d642af86c064e6fff8c877"
server_key_version: "encryptionkey@0"
I20260811 10:12:26.929860 16331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba"
format_stamp: "Formatted at 2026-08-11 10:12:26 on dist-test-slave-7f01"
server_key: "0ae7b119680b9f6a17c3f70c6bbc6772"
server_key_iv: "e12657e662d642af86c064e6fff8c877"
server_key_version: "encryptionkey@0"
I20260811 10:12:26.931190 16331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:26.931960 16346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:26.932199 16331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:26.932317 16331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba"
format_stamp: "Formatted at 2026-08-11 10:12:26 on dist-test-slave-7f01"
server_key: "0ae7b119680b9f6a17c3f70c6bbc6772"
server_key_iv: "e12657e662d642af86c064e6fff8c877"
server_key_version: "encryptionkey@0"
I20260811 10:12:26.932402 16331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:26.941056 16331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:26.942826 16331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:26.942992 16331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:26.947190 16331 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:40393
I20260811 10:12:26.947217 16444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:40393 every 8 connection(s)
I20260811 10:12:26.947486 16331 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:26.948565 16447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:26.950922 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: Bootstrap starting.
I20260811 10:12:26.951553 16447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:26.951930 16447 log.cc:824] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:26.952389 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16331
I20260811 10:12:26.952503 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:26.952703 9144 external_mini_cluster.cc:1428] Setting key 20cd9b334221b5403de9dd2641964d58
I20260811 10:12:26.952704 16447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: No bootstrap required, opened a new log
I20260811 10:12:26.955533 16447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } }
I20260811 10:12:26.955693 16447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:26.955761 16447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcab3a14d8b4bde88ce7fedf38f92ba, State: Initialized, Role: FOLLOWER
I20260811 10:12:26.955940 16447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } }
I20260811 10:12:26.956029 16447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:26.956075 16447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:26.956125 16447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:26.956673 16447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } }
I20260811 10:12:26.956806 16447 leader_election.cc:302] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bcab3a14d8b4bde88ce7fedf38f92ba; no voters:
I20260811 10:12:26.957013 16447 leader_election.cc:288] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:26.957039 16452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:26.957150 16452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [term 1 LEADER]: Becoming Leader. State: Replica: 4bcab3a14d8b4bde88ce7fedf38f92ba, State: Running, Role: LEADER
I20260811 10:12:26.957235 16452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } }
I20260811 10:12:26.957314 16447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:26.957589 16452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } } }
I20260811 10:12:26.957738 16452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:26.959120 16455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bcab3a14d8b4bde88ce7fedf38f92ba. Latest consensus state: current_term: 1 leader_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcab3a14d8b4bde88ce7fedf38f92ba" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40393 } } }
I20260811 10:12:26.959273 16455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:26.959540 16463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 10:12:26 dist-test-slave-7f01 krb5kdc[16303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:26.960183 16463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:26.961759 16463 catalog_manager.cc:1347] Generated new cluster ID: 56e9b75946ea42d7b75741e4fc64b81c
I20260811 10:12:26.961814 16463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:26.963936 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:26.953220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.8.238.62:40393)
0811 10:12:26.953396 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:26.953405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:26.953439 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:12:26.954104 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:26.954111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:26.954158 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:26.954310 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:26.955076 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:26.956113 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:26.956896 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:26.957505 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:26.957627 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:26.960296 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:26.960299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:26.960300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:26.960312 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:26.962305 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:26.962696 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:26.962699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:26.962700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:26.962803 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:26.963015 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:26.963016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:26.963017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:26.963313 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:26.963475 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:26.963558 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:12:26.963698 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260811 10:12:26.986661 16463 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:26.987388 16463 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:26.996372 16463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: Generated new TSK 0
I20260811 10:12:26.996569 16463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:27.035379 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:27.121562 16490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:27.121778 16490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:27.121802 16490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:27.123027 16490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:27.123070 16490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:27.123098 16490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:27.124718 16490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:27.124989 16490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:27.125231 16490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:27.130399 16498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:27.130368 16500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:27.130576 16490 server_base.cc:1034] running on GCE node
W20260811 10:12:27.130373 16503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:27.130913 16490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:27.131166 16490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:27.132332 16490 hybrid_clock.cc:648] HybridClock initialized: now 1786443147132316 us; error 49 us; skew 500 ppm
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:27.135015 16490 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:27.135432 16490 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:27.135514 16490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:27.135667 16490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:27.136749 16490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "8227c7f9165e41b591f2fac99f2106c8"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "23ae2f5d66e35b57d09b3c8f960dfd58"
server_key_iv: "9a6267bc82dfe4008fcde60f3b6b162f"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.137122 16490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "8227c7f9165e41b591f2fac99f2106c8"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "23ae2f5d66e35b57d09b3c8f960dfd58"
server_key_iv: "9a6267bc82dfe4008fcde60f3b6b162f"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.138420 16490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:27.139160 16515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.139365 16490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:27.139482 16490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "8227c7f9165e41b591f2fac99f2106c8"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "23ae2f5d66e35b57d09b3c8f960dfd58"
server_key_iv: "9a6267bc82dfe4008fcde60f3b6b162f"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.139559 16490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:27.176795 16490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:27.178817 16490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:27.178999 16490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:27.179255 16490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:27.179605 16490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:27.179663 16490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.179701 16490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:27.179769 16490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.185866 16490 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:41069
I20260811 10:12:27.185913 16715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:41069 every 8 connection(s)
I20260811 10:12:27.186185 16490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:27.191452 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16490
I20260811 10:12:27.191579 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:27.191840 9144 external_mini_cluster.cc:1428] Setting key 098405774cc9717dfab116a5bc27d772
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:27.196360 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.186989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:58841 (local address 127.8.238.62:40393)
0811 10:12:27.187054 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.187056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.187260 (+ 204us) server_negotiation.cc:407] Connection header received
0811 10:12:27.187795 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.187796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.187807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.187877 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:27.188803 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.189359 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.190354 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.190548 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.190598 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.193059 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:27.193061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:27.193061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:27.193067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:27.194832 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.195226 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.195227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.195228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.195311 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.195552 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.195553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.195554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.195711 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:27.195828 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.196160 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.196265 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:12:27.196547 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.186956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40393 (local address 127.8.238.1:58841)
0811 10:12:27.187130 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.187174 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.187670 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.187941 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.187945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.188205 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:27.188721 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.188727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.189448 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.189449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.190278 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.190280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.190620 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.190621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.190723 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.191164 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:27.191172 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:27.192954 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:27.194920 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.194927 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.194928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.195148 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.195380 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.195380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.195381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.195487 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.195824 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:27.195829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:27.195962 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.196123 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.196296 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260811 10:12:27.197050 16716 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40393
I20260811 10:12:27.197157 16716 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:27.197307 16716 heartbeater.cc:511] Master 127.8.238.62:40393 requested a full tablet report, sending...
I20260811 10:12:27.197901 16385 ts_manager.cc:194] Registered new tserver with Master: 8227c7f9165e41b591f2fac99f2106c8 (127.8.238.1:41069)
I20260811 10:12:27.198534 16385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:58841
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:27.218622 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:27.301932 16730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:27.302150 16730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:27.302173 16730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:27.303359 16730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:27.303402 16730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:27.303463 16730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:27.304989 16730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:27.305243 16730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:27.305486 16730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:27.310526 16736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:27.310551 16737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:27.310526 16741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:27.310771 16730 server_base.cc:1034] running on GCE node
I20260811 10:12:27.311002 16730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:27.311245 16730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:27.312415 16730 hybrid_clock.cc:648] HybridClock initialized: now 1786443147312391 us; error 40 us; skew 500 ppm
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:27.315021 16730 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:27.315308 16730 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:27.315351 16730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:27.315492 16730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:27.316615 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "5af1a897b11d4347b48a9383edabe90c"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "e663ae5e68a2fae0c4d0698dde0c253c"
server_key_iv: "8a8a95861ab589dcc673c164100880ca"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.316987 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "5af1a897b11d4347b48a9383edabe90c"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "e663ae5e68a2fae0c4d0698dde0c253c"
server_key_iv: "8a8a95861ab589dcc673c164100880ca"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.318301 16730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:27.319060 16749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.319236 16730 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:27.319329 16730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "5af1a897b11d4347b48a9383edabe90c"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "e663ae5e68a2fae0c4d0698dde0c253c"
server_key_iv: "8a8a95861ab589dcc673c164100880ca"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.319384 16730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:27.328128 16730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:27.329975 16730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:27.330084 16730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:27.330259 16730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:27.330510 16730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:27.330541 16730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.330562 16730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:27.330574 16730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.336851 16730 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:37771
I20260811 10:12:27.336884 16932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:37771 every 8 connection(s)
I20260811 10:12:27.337180 16730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:27.343411 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16730
I20260811 10:12:27.343528 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:27.343724 9144 external_mini_cluster.cc:1428] Setting key cc4984744288d0caeefa43a7f4260f16
I20260811 10:12:27.346817 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.337857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:40013 (local address 127.8.238.62:40393)
0811 10:12:27.337900 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.337902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.338105 (+ 203us) server_negotiation.cc:407] Connection header received
0811 10:12:27.338636 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.338638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.338652 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.338717 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:27.339664 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.340281 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.341288 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.341511 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.341589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.344030 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:27.344032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:27.344032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:27.344040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:27.345406 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.345819 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.345820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.345820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.345881 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.346144 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.346145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.346145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.346260 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:27.346341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.346620 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.346711 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:12:27.347043 16936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.337839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40393 (local address 127.8.238.2:40013)
0811 10:12:27.337989 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.338024 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.338529 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.338784 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.338788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.339070 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:27.339559 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.339565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.340381 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.340383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.341194 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.341197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.341597 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.341598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.341744 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.342317 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:27.342327 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:27.343931 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:27.345494 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.345502 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.345504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.345728 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.345947 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.345948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.345949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.346066 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.346337 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:27.346345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:27.346485 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.346586 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.346742 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260811 10:12:27.347523 16933 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40393
I20260811 10:12:27.347631 16933 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:27.347815 16933 heartbeater.cc:511] Master 127.8.238.62:40393 requested a full tablet report, sending...
I20260811 10:12:27.348171 16385 ts_manager.cc:194] Registered new tserver with Master: 5af1a897b11d4347b48a9383edabe90c (127.8.238.2:37771)
I20260811 10:12:27.348558 16385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:40013
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:27.370813 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:27.453223 16940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:27.453420 16940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:27.453444 16940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:27.454587 16940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:27.454629 16940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:27.454654 16940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:27.456182 16940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:27.456362 16940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:27.456530 16940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:27.461676 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:27.461648 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:27.461686 16949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:27.461877 16940 server_base.cc:1034] running on GCE node
I20260811 10:12:27.462087 16940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:27.462288 16940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:27.463433 16940 hybrid_clock.cc:648] HybridClock initialized: now 1786443147463421 us; error 27 us; skew 500 ppm
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:27.466141 16940 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:27.466480 16940 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:27.466554 16940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:27.466704 16940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:27.467814 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "73060624617c47df86e42b03fb7d9858"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "834b23822fd3eac41d19d7201565602d"
server_key_iv: "edbd5d41d1a303ff3a1b1ed14b2dda65"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.468200 16940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "73060624617c47df86e42b03fb7d9858"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "834b23822fd3eac41d19d7201565602d"
server_key_iv: "edbd5d41d1a303ff3a1b1ed14b2dda65"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.469494 16940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:27.470194 16963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.470371 16940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:27.470484 16940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "73060624617c47df86e42b03fb7d9858"
format_stamp: "Formatted at 2026-08-11 10:12:27 on dist-test-slave-7f01"
server_key: "834b23822fd3eac41d19d7201565602d"
server_key_iv: "edbd5d41d1a303ff3a1b1ed14b2dda65"
server_key_version: "encryptionkey@0"
I20260811 10:12:27.470557 16940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:27.485386 16940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:27.487224 16940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:27.487389 16940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:27.487622 16940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:27.487998 16940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:27.488054 16940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.488087 16940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:27.488118 16940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:27.494029 16940 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:37191
I20260811 10:12:27.494107 17140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:37191 every 8 connection(s)
I20260811 10:12:27.494386 16940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:27.495576 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16940
I20260811 10:12:27.495679 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:27.495905 9144 external_mini_cluster.cc:1428] Setting key a96109a805f9c0ee3733fd0a3f4f4a07
Aug 11 10:12:27 dist-test-slave-7f01 krb5kdc[16303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:27.504154 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.495130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:48353 (local address 127.8.238.62:40393)
0811 10:12:27.495222 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.495225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.495395 (+ 170us) server_negotiation.cc:407] Connection header received
0811 10:12:27.496020 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.496022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.496035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.496093 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:27.497343 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.497865 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.498876 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.499102 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.499177 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.501283 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:27.501285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:27.501285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:27.501293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:27.502697 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.503119 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.503120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.503120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.503181 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.503429 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.503430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.503431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.503549 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:27.503628 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.503973 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.504063 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:12:27.504316 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.495095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40393 (local address 127.8.238.3:48353)
0811 10:12:27.495278 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.495312 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.495905 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.496160 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.496166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.496579 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:27.497247 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.497253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.497963 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.497964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.498779 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.498781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.499199 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.499199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.499326 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.499842 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:27.499850 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:27.501183 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:27.502798 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.502805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.502807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.503030 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.503248 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:27.503249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:27.503249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:27.503361 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:27.503619 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:27.503626 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:27.503813 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.503930 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.504086 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260811 10:12:27.504802 17141 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40393
I20260811 10:12:27.504884 17141 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:27.505000 17141 heartbeater.cc:511] Master 127.8.238.62:40393 requested a full tablet report, sending...
I20260811 10:12:27.505342 16385 ts_manager.cc:194] Registered new tserver with Master: 73060624617c47df86e42b03fb7d9858 (127.8.238.3:37191)
I20260811 10:12:27.505834 16385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:48353
I20260811 10:12:27.507886 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:12:27.515035 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.509296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.8.238.62:40393)
0811 10:12:27.509366 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.509368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.509495 (+ 127us) server_negotiation.cc:407] Connection header received
0811 10:12:27.509541 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.509542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.509551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.509595 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:27.510163 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.510608 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.511354 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.511527 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.511575 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.512730 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:27.512731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:27.512731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:27.512737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:27.514082 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.514390 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.514391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.514391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.514439 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:27.514601 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:27.514601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:27.514601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:27.514726 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:27.514799 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.514911 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.514978 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:12:27.516548 16385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:27.518155 16385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:27.525175 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41069 (local address 127.0.0.1:54386)
0811 10:12:27.520699 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.520742 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.520835 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.521218 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.521221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.521224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:27.521296 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.521298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.522663 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.522664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.524063 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.524067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524814 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.524913 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.524954 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.525025 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.525091 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":46,"thread_start_us":75,"threads_started":1}
I20260811 10:12:27.525182 17166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:37771 (local address 127.0.0.1:42904)
0811 10:12:27.520981 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.520992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.521046 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.521526 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.521526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.521528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:27.521575 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.521577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.522774 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.522775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.524033 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.524037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524974 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.525038 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.525041 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.525098 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.525126 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260811 10:12:27.525549 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.8.238.1:41069)
0811 10:12:27.520999 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.521003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.521024 (+ 21us) server_negotiation.cc:407] Connection header received
0811 10:12:27.521092 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.521094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.521111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.521198 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:27.521390 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.522569 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524177 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524697 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524803 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.524871 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.525394 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.525453 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":100,"threads_started":1}
I20260811 10:12:27.525552 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.8.238.2:37771)
0811 10:12:27.521371 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.521376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.521387 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:27.521428 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.521430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.521447 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.521527 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:27.521620 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.522682 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524110 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524842 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524950 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.525007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.525113 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.525152 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":86,"threads_started":1}
I20260811 10:12:27.526392 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:37191 (local address 127.0.0.1:46036)
0811 10:12:27.521118 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.521126 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.521171 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.523116 (+ 1945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.523117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.523120 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:27.523174 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.523176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524171 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.524171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.525209 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.525212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.526149 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.526150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.526214 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.526219 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.526276 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.526322 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260811 10:12:27.526407 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.520948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.8.238.3:37191)
0811 10:12:27.521120 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.521124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.522680 (+ 1556us) server_negotiation.cc:407] Connection header received
0811 10:12:27.522728 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.522730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.522748 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.523014 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:27.523258 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.524084 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.525308 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.526038 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.526144 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.526199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.526294 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.526332 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260811 10:12:27.526292 16597 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee (DEFAULT_TABLE table=test-table [id=3dc40772e680447580499fff1eb74287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:27.526366 16830 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee (DEFAULT_TABLE table=test-table [id=3dc40772e680447580499fff1eb74287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:27.526677 16830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:27.526678 16597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:27.527278 17034 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee (DEFAULT_TABLE table=test-table [id=3dc40772e680447580499fff1eb74287]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:27.527590 17034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:27.530042 17172 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Bootstrap starting.
I20260811 10:12:27.530522 17174 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Bootstrap starting.
I20260811 10:12:27.531045 17172 tablet_bootstrap.cc:654] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:27.531107 17174 tablet_bootstrap.cc:654] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:27.531622 17174 log.cc:824] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:27.531633 17172 log.cc:824] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:27.531893 17173 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Bootstrap starting.
I20260811 10:12:27.532524 17174 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: No bootstrap required, opened a new log
I20260811 10:12:27.532549 17173 tablet_bootstrap.cc:654] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:27.532618 17172 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: No bootstrap required, opened a new log
I20260811 10:12:27.532652 17174 ts_tablet_manager.cc:1397] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:27.532697 17172 ts_tablet_manager.cc:1397] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:12:27.533177 17173 log.cc:824] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:27.534111 17173 tablet_bootstrap.cc:492] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: No bootstrap required, opened a new log
I20260811 10:12:27.534189 17173 ts_tablet_manager.cc:1397] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:27.534569 17172 raft_consensus.cc:348] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.534767 17172 raft_consensus.cc:374] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:27.534813 17172 raft_consensus.cc:729] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73060624617c47df86e42b03fb7d9858, State: Initialized, Role: FOLLOWER
I20260811 10:12:27.534845 17174 raft_consensus.cc:348] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.535019 17174 raft_consensus.cc:374] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:27.534977 17172 consensus_queue.cc:260] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.535065 17174 raft_consensus.cc:729] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af1a897b11d4347b48a9383edabe90c, State: Initialized, Role: FOLLOWER
I20260811 10:12:27.535197 17174 consensus_queue.cc:260] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.535262 17172 ts_tablet_manager.cc:1428] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:27.535385 17141 heartbeater.cc:503] Master 127.8.238.62:40393 was elected leader, sending a full tablet report...
I20260811 10:12:27.535473 17174 ts_tablet_manager.cc:1428] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:27.535604 16933 heartbeater.cc:503] Master 127.8.238.62:40393 was elected leader, sending a full tablet report...
I20260811 10:12:27.536428 17173 raft_consensus.cc:348] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.536592 17173 raft_consensus.cc:374] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:27.536644 17173 raft_consensus.cc:729] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8227c7f9165e41b591f2fac99f2106c8, State: Initialized, Role: FOLLOWER
I20260811 10:12:27.536780 17173 consensus_queue.cc:260] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.537003 17173 ts_tablet_manager.cc:1428] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:27.537067 16716 heartbeater.cc:503] Master 127.8.238.62:40393 was elected leader, sending a full tablet report...
W20260811 10:12:27.587859 16935 tablet.cc:2369] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:12:27.686987 16718 tablet.cc:2369] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:12:27.744992 17142 tablet.cc:2369] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:27.990404 17180 raft_consensus.cc:482] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:27.990590 17180 raft_consensus.cc:504] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.991043 17180 leader_election.cc:288] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8227c7f9165e41b591f2fac99f2106c8 (127.8.238.1:41069), 73060624617c47df86e42b03fb7d9858 (127.8.238.3:37191)
I20260811 10:12:27.994470 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.991328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:37151 (local address 127.8.238.1:41069)
0811 10:12:27.991382 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.991384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.991519 (+ 135us) server_negotiation.cc:407] Connection header received
0811 10:12:27.991606 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.991608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.991622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.991691 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:27.991937 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.992710 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.993697 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.994096 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.994187 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.994199 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.994349 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.994391 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:12:27.994496 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.991251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:37191 (local address 127.8.238.2:52193)
0811 10:12:27.991410 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.991423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.991495 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.991697 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.991699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.991701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:27.991810 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.991812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.992837 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.992839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.993609 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.993611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.994194 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.994195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.994275 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.994278 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.994362 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.994404 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260811 10:12:27.994495 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.991379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:52193 (local address 127.8.238.3:37191)
0811 10:12:27.991439 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:27.991442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:27.991473 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:12:27.991594 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:27.991595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:27.991615 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:27.991673 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:27.991886 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.992711 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.993723 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.994100 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.994186 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.994198 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:27.994380 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:12:27.994434 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:27.994500 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:27.991252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41069 (local address 127.8.238.2:37151)
0811 10:12:27.991433 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:27.991443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:27.991535 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:27.991710 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:27.991711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:27.991713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:27.991824 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.991826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.992800 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.992801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.993591 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:27.993593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:27.994189 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:27.994190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:27.994268 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:27.994271 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:27.994315 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 10:12:27.994370 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":23,"thread_start_us":116,"threads_started":1}
I20260811 10:12:27.995057 17069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2cdf926a1347e1ae724c5fbc8fa0ee" candidate_uuid: "5af1a897b11d4347b48a9383edabe90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73060624617c47df86e42b03fb7d9858" is_pre_election: true
I20260811 10:12:27.995065 16640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2cdf926a1347e1ae724c5fbc8fa0ee" candidate_uuid: "5af1a897b11d4347b48a9383edabe90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8227c7f9165e41b591f2fac99f2106c8" is_pre_election: true
I20260811 10:12:27.995232 17069 raft_consensus.cc:2454] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af1a897b11d4347b48a9383edabe90c in term 0.
I20260811 10:12:27.995250 16640 raft_consensus.cc:2454] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af1a897b11d4347b48a9383edabe90c in term 0.
I20260811 10:12:27.995530 16750 leader_election.cc:302] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5af1a897b11d4347b48a9383edabe90c, 73060624617c47df86e42b03fb7d9858, 8227c7f9165e41b591f2fac99f2106c8; no voters:
I20260811 10:12:27.995720 17180 raft_consensus.cc:2781] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:27.995824 17180 raft_consensus.cc:482] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:27.995857 17180 raft_consensus.cc:3037] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:27.996696 17180 raft_consensus.cc:504] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.996855 17180 leader_election.cc:288] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [CANDIDATE]: Term 1 election: Requested vote from peers 8227c7f9165e41b591f2fac99f2106c8 (127.8.238.1:41069), 73060624617c47df86e42b03fb7d9858 (127.8.238.3:37191)
I20260811 10:12:27.997030 16640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2cdf926a1347e1ae724c5fbc8fa0ee" candidate_uuid: "5af1a897b11d4347b48a9383edabe90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8227c7f9165e41b591f2fac99f2106c8"
I20260811 10:12:27.997041 17069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2cdf926a1347e1ae724c5fbc8fa0ee" candidate_uuid: "5af1a897b11d4347b48a9383edabe90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73060624617c47df86e42b03fb7d9858"
I20260811 10:12:27.997108 16640 raft_consensus.cc:3037] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:27.997136 17069 raft_consensus.cc:3037] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:27.997844 16640 raft_consensus.cc:2454] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af1a897b11d4347b48a9383edabe90c in term 1.
I20260811 10:12:27.997870 17069 raft_consensus.cc:2454] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af1a897b11d4347b48a9383edabe90c in term 1.
I20260811 10:12:27.998023 16750 leader_election.cc:302] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af1a897b11d4347b48a9383edabe90c, 8227c7f9165e41b591f2fac99f2106c8; no voters:
I20260811 10:12:27.998131 17180 raft_consensus.cc:2781] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:27.998265 17180 raft_consensus.cc:686] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 1 LEADER]: Becoming Leader. State: Replica: 5af1a897b11d4347b48a9383edabe90c, State: Running, Role: LEADER
I20260811 10:12:27.998337 17180 consensus_queue.cc:237] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } }
I20260811 10:12:27.999190 16385 catalog_manager.cc:5495] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af1a897b11d4347b48a9383edabe90c (127.8.238.2). New cstate: current_term: 1 leader_uuid: "5af1a897b11d4347b48a9383edabe90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1a897b11d4347b48a9383edabe90c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:28.074815 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.072192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.8.238.2:37771)
0811 10:12:28.072322 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.072324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.072357 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:12:28.072437 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.072438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.072459 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.072538 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:28.072664 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:28.073190 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:28.073939 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:28.074140 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:28.074193 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:28.074423 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:28.074540 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:28.074636 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.074686 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.074711 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260811 10:12:28.076820 17069 raft_consensus.cc:1261] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 1 FOLLOWER]: Refusing update from remote peer 5af1a897b11d4347b48a9383edabe90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:28.076838 16640 raft_consensus.cc:1261] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 1 FOLLOWER]: Refusing update from remote peer 5af1a897b11d4347b48a9383edabe90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:28.077196 17180 consensus_queue.cc:1035] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8227c7f9165e41b591f2fac99f2106c8" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:28.077368 17186 consensus_queue.cc:1035] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "73060624617c47df86e42b03fb7d9858" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 37191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:28.080010 17193 mvcc.cc:205] Tried to move back new op lower bound from 7317271134519549952 to 7317271134201802752. Current Snapshot: MvccSnapshot[applied={T|T < 7317271134519549952}]
I20260811 10:12:28.080127 17198 mvcc.cc:205] Tried to move back new op lower bound from 7317271134519549952 to 7317271134201802752. Current Snapshot: MvccSnapshot[applied={T|T < 7317271134519549952}]
I20260811 10:12:28.081064 17196 mvcc.cc:205] Tried to move back new op lower bound from 7317271134519549952 to 7317271134201802752. Current Snapshot: MvccSnapshot[applied={T|T < 7317271134519549952}]
I20260811 10:12:28.082491 16383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:28.082610 16383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:28.083415 16383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bcab3a14d8b4bde88ce7fedf38f92ba: Sending DeleteTablet for 3 replicas of tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee
I20260811 10:12:28.083760 16830 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:42904
I20260811 10:12:28.083808 17034 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:46036
I20260811 10:12:28.083811 16597 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:54386
I20260811 10:12:28.084040 17210 tablet_replica.cc:330] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8: stopping tablet replica
I20260811 10:12:28.084128 17208 tablet_replica.cc:330] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: stopping tablet replica
I20260811 10:12:28.084300 17210 raft_consensus.cc:2229] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 8227c7f9165e41b591f2fac99f2106c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:28.084376 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16490
I20260811 10:12:28.087257 17209 tablet_replica.cc:330] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: stopping tablet replica
I20260811 10:12:28.087353 17209 raft_consensus.cc:2229] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:28.087491 17209 raft_consensus.cc:2258] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:28.090236 17209 ts_tablet_manager.cc:1905] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:28.092608 17209 ts_tablet_manager.cc:1918] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:28.092707 17209 log.cc:1196] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/0e2cdf926a1347e1ae724c5fbc8fa0ee
I20260811 10:12:28.093027 17209 ts_tablet_manager.cc:1939] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 73060624617c47df86e42b03fb7d9858: Deleting consensus metadata
I20260811 10:12:28.093447 16366 catalog_manager.cc:4841] TS 73060624617c47df86e42b03fb7d9858 (127.8.238.3:37191): tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee (table test-table [id=3dc40772e680447580499fff1eb74287]) successfully deleted
I20260811 10:12:28.084271 17208 raft_consensus.cc:2229] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:28.101923 17208 raft_consensus.cc:2258] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:28.103008 17208 ts_tablet_manager.cc:1905] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:28.105094 17208 ts_tablet_manager.cc:1918] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:28.105162 17208 log.cc:1196] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/0e2cdf926a1347e1ae724c5fbc8fa0ee
I20260811 10:12:28.105374 17208 ts_tablet_manager.cc:1939] T 0e2cdf926a1347e1ae724c5fbc8fa0ee P 5af1a897b11d4347b48a9383edabe90c: Deleting consensus metadata
I20260811 10:12:28.108820 16366 catalog_manager.cc:4841] TS 5af1a897b11d4347b48a9383edabe90c (127.8.238.2:37771): tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee (table test-table [id=3dc40772e680447580499fff1eb74287]) successfully deleted
W20260811 10:12:28.121337 16360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:41069 (error 108)
I20260811 10:12:28.121591 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16730
I20260811 10:12:28.124020 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.121783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41069 (local address 127.0.0.1:54396)
0811 10:12:28.123905 (+ 2122us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.123962 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2100,"thread_start_us":49,"threads_started":1}
W20260811 10:12:28.124219 16360 catalog_manager.cc:4584] TS 8227c7f9165e41b591f2fac99f2106c8 (127.8.238.1:41069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e2cdf926a1347e1ae724c5fbc8fa0ee: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41069: connect: Connection refused (error 111)
I20260811 10:12:28.128371 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16940
I20260811 10:12:28.138556 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16331
2026-08-11T10:12:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2708 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T10:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:28Z Disabled control of system clock
I20260811 10:12:28.170073 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46399
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:46399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:12:28.252491 17221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:28.252688 17221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:28.252709 17221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:28.253767 17221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:28.253798 17221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:28.253811 17221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:28.253824 17221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:28.253836 17221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:28.255077 17221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:46399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:28.255257 17221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:28.255491 17221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:28.260231 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.260309 17230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:28.260275 17221 server_base.cc:1034] running on GCE node
W20260811 10:12:28.260241 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:28.260753 17221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:28.261066 17221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:28.262235 17221 hybrid_clock.cc:648] HybridClock initialized: now 1786443148262209 us; error 55 us; skew 500 ppm
I20260811 10:12:28.263564 17221 webserver.cc:460] Webserver started at http://127.8.238.62:42449/ using document root <none> and password file <none>
I20260811 10:12:28.263873 17221 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:28.263945 17221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:28.264101 17221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:28.265208 17221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "75b4cd9062a64c0a8be8c08fd852a835"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "25796ca62df93caaf4ee1ca4fde69f50"
server_key_iv: "96a3f3da031a6c616f0eebc1d3a21dc3"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.265601 17221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "75b4cd9062a64c0a8be8c08fd852a835"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "25796ca62df93caaf4ee1ca4fde69f50"
server_key_iv: "96a3f3da031a6c616f0eebc1d3a21dc3"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.266913 17221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:28.267603 17247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.267819 17221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:28.267925 17221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "75b4cd9062a64c0a8be8c08fd852a835"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "25796ca62df93caaf4ee1ca4fde69f50"
server_key_iv: "96a3f3da031a6c616f0eebc1d3a21dc3"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.268011 17221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:28.271260 17221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:28.271538 17221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:28.271683 17221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:28.275799 17221 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:46399
I20260811 10:12:28.275825 17337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:46399 every 8 connection(s)
I20260811 10:12:28.276151 17221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:28.276775 17339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:28.279572 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: Bootstrap starting.
I20260811 10:12:28.280193 17339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:28.280512 17339 log.cc:824] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:28.281172 17339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: No bootstrap required, opened a new log
I20260811 10:12:28.282738 17339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } }
I20260811 10:12:28.282871 17339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:28.282894 17339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b4cd9062a64c0a8be8c08fd852a835, State: Initialized, Role: FOLLOWER
I20260811 10:12:28.283069 17339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } }
I20260811 10:12:28.283133 17339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:28.283154 17339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:28.283218 17339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:28.283807 17339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } }
I20260811 10:12:28.283926 17339 leader_election.cc:302] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b4cd9062a64c0a8be8c08fd852a835; no voters:
I20260811 10:12:28.284125 17339 leader_election.cc:288] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:28.284158 17346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:28.284209 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 17221
I20260811 10:12:28.284317 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:28.284328 17346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [term 1 LEADER]: Becoming Leader. State: Replica: 75b4cd9062a64c0a8be8c08fd852a835, State: Running, Role: LEADER
I20260811 10:12:28.284410 17339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:28.284422 17346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } }
I20260811 10:12:28.284521 9144 external_mini_cluster.cc:1428] Setting key 0f53468c07d31680dec4368ed7ccb57a
I20260811 10:12:28.284757 17346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b4cd9062a64c0a8be8c08fd852a835. Latest consensus state: current_term: 1 leader_uuid: "75b4cd9062a64c0a8be8c08fd852a835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } } }
I20260811 10:12:28.284839 17346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:28.284984 17347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b4cd9062a64c0a8be8c08fd852a835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b4cd9062a64c0a8be8c08fd852a835" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46399 } } }
I20260811 10:12:28.285045 17347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:28.285583 17354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:28.286088 17354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:28.287590 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.285139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.8.238.62:46399)
0811 10:12:28.285373 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.285381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.285412 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:12:28.285623 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.285630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.285731 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.285875 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.287131 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.287136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.287136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.287140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.287180 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.287196 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.287274 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.287356 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.287409 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260811 10:12:28.287909 17354 catalog_manager.cc:1347] Generated new cluster ID: e3e7b6134f3a470485ac25a2380ff621
I20260811 10:12:28.287973 17354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:28.304877 17354 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:28.305087 17354 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:28.307811 17354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: Generated new TSK 0
I20260811 10:12:28.308013 17354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:12:28.311368 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--builtin_ntp_servers=127.8.238.20:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:12:28.414687 17381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:28.414933 17381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:28.414973 17381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:28.416173 17381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:28.416234 17381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:28.416275 17381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:28.417574 17381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:28.417845 17381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:28.418140 17381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:28.423106 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.423095 17393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.423251 17390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:28.423486 17381 server_base.cc:1034] running on GCE node
I20260811 10:12:28.423717 17381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:28.424007 17381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:28.425172 17381 hybrid_clock.cc:648] HybridClock initialized: now 1786443148425160 us; error 30 us; skew 500 ppm
I20260811 10:12:28.426523 17381 webserver.cc:460] Webserver started at http://127.8.238.1:41833/ using document root <none> and password file <none>
I20260811 10:12:28.426800 17381 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:28.426878 17381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:28.427052 17381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:28.428201 17381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "351d8b9947794774bb9476f59273a4fd"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "aba38310e9ac6dd1ee5ebec3fd0f1b8d"
server_key_iv: "89a4c02ecd11aba44388c772d2d4234d"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.428589 17381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "351d8b9947794774bb9476f59273a4fd"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "aba38310e9ac6dd1ee5ebec3fd0f1b8d"
server_key_iv: "89a4c02ecd11aba44388c772d2d4234d"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.429976 17381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:28.431069 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.431272 17381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:28.431398 17381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "351d8b9947794774bb9476f59273a4fd"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "aba38310e9ac6dd1ee5ebec3fd0f1b8d"
server_key_iv: "89a4c02ecd11aba44388c772d2d4234d"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.431486 17381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:28.435293 17381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:28.435611 17381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:28.435792 17381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:28.436057 17381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:28.436398 17381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:28.436458 17381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.436502 17381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:28.436532 17381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.443250 17381 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:33543
I20260811 10:12:28.443328 17601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:33543 every 8 connection(s)
I20260811 10:12:28.443683 17381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:28.446993 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.444567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:37119 (local address 127.8.238.62:46399)
0811 10:12:28.444629 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.444631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.444893 (+ 262us) server_negotiation.cc:407] Connection header received
0811 10:12:28.445441 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.445443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.445451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.445503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.446702 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.446704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.446705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.446709 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.446727 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.446731 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.446799 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.446905 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.446923 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:28.447216 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.444530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46399 (local address 127.8.238.1:37119)
0811 10:12:28.444775 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.444808 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.445321 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.445574 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.445584 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.445593 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.446548 (+ 955us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.446549 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.446577 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.446605 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.446805 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.446812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.446815 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.446915 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.446977 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":93,"threads_started":1}
I20260811 10:12:28.447834 17602 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46399
I20260811 10:12:28.447921 17602 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:28.448104 17602 heartbeater.cc:511] Master 127.8.238.62:46399 requested a full tablet report, sending...
I20260811 10:12:28.448191 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 17381
I20260811 10:12:28.448311 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:28.448539 9144 external_mini_cluster.cc:1428] Setting key 8189a93ac38647fbc47494e9d72531a7
I20260811 10:12:28.448684 17276 ts_manager.cc:194] Registered new tserver with Master: 351d8b9947794774bb9476f59273a4fd (127.8.238.1:33543)
I20260811 10:12:28.449746 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--builtin_ntp_servers=127.8.238.20:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 10:12:28.536598 17608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:28.536788 17608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:28.536810 17608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:28.537886 17608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:28.537920 17608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:28.537945 17608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:28.539197 17608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:28.539395 17608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:28.539707 17608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:28.544998 17617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.545025 17616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.545037 17620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:28.545249 17608 server_base.cc:1034] running on GCE node
I20260811 10:12:28.545483 17608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:28.545758 17608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:28.546922 17608 hybrid_clock.cc:648] HybridClock initialized: now 1786443148546915 us; error 35 us; skew 500 ppm
I20260811 10:12:28.548261 17608 webserver.cc:460] Webserver started at http://127.8.238.2:40381/ using document root <none> and password file <none>
I20260811 10:12:28.548516 17608 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:28.548590 17608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:28.548741 17608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:28.549760 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "420c735f72574cc482a430759d1a611b"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "2cbfc25b1ea06b5ecae205d140a83958"
server_key_iv: "ca7f69209503f5c5d7d8ff699e6fffdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.550122 17608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "420c735f72574cc482a430759d1a611b"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "2cbfc25b1ea06b5ecae205d140a83958"
server_key_iv: "ca7f69209503f5c5d7d8ff699e6fffdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.551447 17608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:28.552234 17627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.552400 17608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.552515 17608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "420c735f72574cc482a430759d1a611b"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "2cbfc25b1ea06b5ecae205d140a83958"
server_key_iv: "ca7f69209503f5c5d7d8ff699e6fffdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.552597 17608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:28.555869 17608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:28.556136 17608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:28.556272 17608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:28.556496 17608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:28.556811 17608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:28.556874 17608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.556911 17608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:28.556938 17608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.562918 17608 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:42857
I20260811 10:12:28.562975 17809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:42857 every 8 connection(s)
I20260811 10:12:28.563278 17608 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:28.565531 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 17608
I20260811 10:12:28.565645 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:28.565864 9144 external_mini_cluster.cc:1428] Setting key 0695e871348a4174e0c82ffb6a821372
I20260811 10:12:28.566529 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.564020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:38253 (local address 127.8.238.62:46399)
0811 10:12:28.564111 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.564114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.564328 (+ 214us) server_negotiation.cc:407] Connection header received
0811 10:12:28.564835 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.564836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.564844 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.564897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.566240 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.566242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.566242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.566246 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.566266 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.566271 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.566327 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.566438 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.566451 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:12:28.566718 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.563970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46399 (local address 127.8.238.2:38253)
0811 10:12:28.564187 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.564228 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.564713 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.564977 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.564981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.564990 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.566077 (+ 1087us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.566078 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.566114 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.566152 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.566347 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.566354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.566357 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.566435 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.566497 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260811 10:12:28.567018 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--builtin_ntp_servers=127.8.238.20:40609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 10:12:28.567220 17811 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46399
I20260811 10:12:28.567315 17811 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:28.567462 17811 heartbeater.cc:511] Master 127.8.238.62:46399 requested a full tablet report, sending...
I20260811 10:12:28.567865 17273 ts_manager.cc:194] Registered new tserver with Master: 420c735f72574cc482a430759d1a611b (127.8.238.2:42857)
W20260811 10:12:28.650445 17815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:28.650645 17815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:28.650665 17815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:28.651890 17815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:28.651932 17815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:28.651957 17815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:28.653312 17815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:28.653564 17815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:28.653855 17815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:28.659056 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.659017 17824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:28.659044 17826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:28.659322 17815 server_base.cc:1034] running on GCE node
I20260811 10:12:28.659564 17815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:28.659828 17815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:28.660979 17815 hybrid_clock.cc:648] HybridClock initialized: now 1786443148660940 us; error 55 us; skew 500 ppm
I20260811 10:12:28.662338 17815 webserver.cc:460] Webserver started at http://127.8.238.3:43097/ using document root <none> and password file <none>
I20260811 10:12:28.662585 17815 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:28.662633 17815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:28.662786 17815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:28.663951 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "ef9a93393d95484c99842f07d6613879"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "a358a5881a012e8d31a24fc42b8ab063"
server_key_iv: "1321acc5d1308370666a7dd515d51b85"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.664345 17815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "ef9a93393d95484c99842f07d6613879"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "a358a5881a012e8d31a24fc42b8ab063"
server_key_iv: "1321acc5d1308370666a7dd515d51b85"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.665668 17815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:28.666399 17835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.666585 17815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:28.666695 17815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "ef9a93393d95484c99842f07d6613879"
format_stamp: "Formatted at 2026-08-11 10:12:28 on dist-test-slave-7f01"
server_key: "a358a5881a012e8d31a24fc42b8ab063"
server_key_iv: "1321acc5d1308370666a7dd515d51b85"
server_key_version: "encryptionkey@0"
I20260811 10:12:28.666782 17815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:28.670161 17815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:28.670421 17815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:28.670570 17815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:28.670776 17815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:28.671144 17815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:28.671200 17815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.671238 17815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:28.671267 17815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:28.678359 17815 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:41133
I20260811 10:12:28.678467 18004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:41133 every 8 connection(s)
I20260811 10:12:28.678766 17815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:28.682331 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.679578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:52821 (local address 127.8.238.62:46399)
0811 10:12:28.679653 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.679656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.679952 (+ 296us) server_negotiation.cc:407] Connection header received
0811 10:12:28.680703 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.680704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.680710 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.680755 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.682115 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.682116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.682116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.682121 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.682137 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.682142 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.682212 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.682261 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.682275 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:12:28.682854 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 17815
I20260811 10:12:28.682960 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:28.683104 9144 external_mini_cluster.cc:1428] Setting key 89728fa2302b04a71b8865ee01a09a49
I20260811 10:12:28.683303 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.679560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46399 (local address 127.8.238.3:52821)
0811 10:12:28.679812 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.679857 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.680581 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.680822 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.680825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.680836 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.681920 (+ 1084us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.681921 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.681961 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.681997 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.682212 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.682215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.682217 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.682267 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.682315 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":115,"threads_started":1}
I20260811 10:12:28.683898 18005 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46399
I20260811 10:12:28.684037 18005 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:28.684181 18005 heartbeater.cc:511] Master 127.8.238.62:46399 requested a full tablet report, sending...
I20260811 10:12:28.684547 17273 ts_manager.cc:194] Registered new tserver with Master: ef9a93393d95484c99842f07d6613879 (127.8.238.3:41133)
I20260811 10:12:28.695050 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:12:28.697753 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.696747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46538 (local address 127.8.238.62:46399)
0811 10:12:28.696808 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.696810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.696896 (+ 86us) server_negotiation.cc:407] Connection header received
0811 10:12:28.696944 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.696945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.696950 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.697000 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.697550 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.697552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.697552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.697556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.697573 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.697577 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.697622 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.697693 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.697706 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:12:28.699299 17273 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:28.701035 17273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:28.705132 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:41133 (local address 127.0.0.1:52294)
0811 10:12:28.703715 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.703793 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.703853 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.704118 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.704127 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.704135 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.704664 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.704665 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.704681 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.704704 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.704953 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.704958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.704961 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.705024 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.705040 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":70,"threads_started":1}
I20260811 10:12:28.705157 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.8.238.3:41133)
0811 10:12:28.703857 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.703863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.703874 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:28.703916 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.703919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.703929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.704021 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.704792 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.704799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.704800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.704804 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.704843 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.704867 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.704900 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.705042 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.705062 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":134,"threads_started":1}
I20260811 10:12:28.705871 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33543 (local address 127.0.0.1:52974)
0811 10:12:28.704061 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.704070 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.704114 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.704804 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.704805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.704808 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.705479 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.705480 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.705486 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.705500 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.705732 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.705733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.705733 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.705793 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.705808 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":83,"threads_started":1}
I20260811 10:12:28.705904 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.8.238.1:33543)
0811 10:12:28.704411 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.704416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.704426 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:28.704455 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.704457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.704467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.704544 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.705597 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.705605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.705605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.705609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.705638 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.705659 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.705735 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.705801 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.705816 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":109,"threads_started":1}
I20260811 10:12:28.705855 17904 tablet_service.cc:1467] Processing CreateTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 (DEFAULT_TABLE table=test-table [id=98c0215ff36d40729940373149f36c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:28.706290 17904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4312a523e8424a2abd2b949fa20a64b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:28.706569 17490 tablet_service.cc:1467] Processing CreateTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 (DEFAULT_TABLE table=test-table [id=98c0215ff36d40729940373149f36c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:28.706943 17490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4312a523e8424a2abd2b949fa20a64b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:28.708287 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:42857 (local address 127.0.0.1:39964)
0811 10:12:28.703936 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.703945 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.703981 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.705253 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.705254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.705256 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.707845 (+ 2589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.707845 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.707853 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.707866 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.708149 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.708150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.708151 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.708211 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.708226 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":86,"threads_started":1}
I20260811 10:12:28.708324 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.703886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.8.238.2:42857)
0811 10:12:28.704422 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.704431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.704439 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:28.704465 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.704466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.704475 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.704577 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.707973 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.707983 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.707984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.707991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.708027 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.708062 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.708133 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.708216 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.708233 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":108,"threads_started":1}
I20260811 10:12:28.709244 17695 tablet_service.cc:1467] Processing CreateTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 (DEFAULT_TABLE table=test-table [id=98c0215ff36d40729940373149f36c50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:28.709594 17695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4312a523e8424a2abd2b949fa20a64b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:28.709589 18033 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Bootstrap starting.
I20260811 10:12:28.710269 18033 tablet_bootstrap.cc:654] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:28.710573 18033 log.cc:824] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:28.710835 18034 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: Bootstrap starting.
I20260811 10:12:28.711555 18034 tablet_bootstrap.cc:654] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:28.711817 18033 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: No bootstrap required, opened a new log
I20260811 10:12:28.711889 18033 ts_tablet_manager.cc:1397] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:28.711916 18034 log.cc:824] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:28.712728 18034 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: No bootstrap required, opened a new log
I20260811 10:12:28.712745 18040 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Bootstrap starting.
I20260811 10:12:28.712812 18034 ts_tablet_manager.cc:1397] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:28.713414 18040 tablet_bootstrap.cc:654] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:28.713768 18040 log.cc:824] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:28.714636 18040 tablet_bootstrap.cc:492] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: No bootstrap required, opened a new log
I20260811 10:12:28.714711 18040 ts_tablet_manager.cc:1397] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:28.714726 18033 raft_consensus.cc:348] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.714766 18034 raft_consensus.cc:348] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.714915 18033 raft_consensus.cc:374] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:28.714939 18034 raft_consensus.cc:374] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:28.714969 18033 raft_consensus.cc:729] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9a93393d95484c99842f07d6613879, State: Initialized, Role: FOLLOWER
I20260811 10:12:28.714985 18034 raft_consensus.cc:729] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351d8b9947794774bb9476f59273a4fd, State: Initialized, Role: FOLLOWER
I20260811 10:12:28.715098 18034 consensus_queue.cc:260] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.715104 18033 consensus_queue.cc:260] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.715374 18033 ts_tablet_manager.cc:1428] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:28.715384 18034 ts_tablet_manager.cc:1428] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:12:28.715401 17602 heartbeater.cc:503] Master 127.8.238.62:46399 was elected leader, sending a full tablet report...
I20260811 10:12:28.715479 18005 heartbeater.cc:503] Master 127.8.238.62:46399 was elected leader, sending a full tablet report...
I20260811 10:12:28.717667 18040 raft_consensus.cc:348] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.717833 18040 raft_consensus.cc:374] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:28.717869 18040 raft_consensus.cc:729] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420c735f72574cc482a430759d1a611b, State: Initialized, Role: FOLLOWER
I20260811 10:12:28.717980 18040 consensus_queue.cc:260] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.718232 18040 ts_tablet_manager.cc:1428] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:12:28.718276 17811 heartbeater.cc:503] Master 127.8.238.62:46399 was elected leader, sending a full tablet report...
I20260811 10:12:28.733784 18042 raft_consensus.cc:482] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:28.733888 18042 raft_consensus.cc:504] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.734314 18042 leader_election.cc:288] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef9a93393d95484c99842f07d6613879 (127.8.238.3:41133), 420c735f72574cc482a430759d1a611b (127.8.238.2:42857)
I20260811 10:12:28.735621 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.734551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:40925 (local address 127.8.238.3:41133)
0811 10:12:28.734594 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.734596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.734606 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:28.734639 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.734640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.734646 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.734729 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.735377 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.735378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.735379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.735383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.735399 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.735403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.735489 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.735538 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.735553 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:28.735621 17606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.734454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:41133 (local address 127.8.238.1:40925)
0811 10:12:28.734521 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.734534 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.734577 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.734750 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.734751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.734754 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.735270 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.735271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.735279 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.735292 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.735488 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.735489 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.735490 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.735539 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.735553 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 10:12:28.735855 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.734507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:42857 (local address 127.8.238.1:49327)
0811 10:12:28.734689 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.734698 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:28.734751 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:28.734930 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:28.734931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:28.734934 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:28.735415 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:12:28.735415 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:28.735423 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:12:28.735436 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:28.735652 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:28.735652 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:28.735653 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:28.735712 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:12:28.735727 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260811 10:12:28.736020 17937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4312a523e8424a2abd2b949fa20a64b4" candidate_uuid: "351d8b9947794774bb9476f59273a4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef9a93393d95484c99842f07d6613879" is_pre_election: true
I20260811 10:12:28.736143 17937 raft_consensus.cc:2454] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351d8b9947794774bb9476f59273a4fd in term 0.
I20260811 10:12:28.736292 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.734570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:49327 (local address 127.8.238.2:42857)
0811 10:12:28.734644 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.734646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.734762 (+ 116us) server_negotiation.cc:407] Connection header received
0811 10:12:28.734811 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.734812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.734817 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.734896 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.735557 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.735558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.735559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.735563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.735580 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.735584 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.735651 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.735860 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.735874 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:12:28.736375 17414 leader_election.cc:302] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351d8b9947794774bb9476f59273a4fd, ef9a93393d95484c99842f07d6613879; no voters:
I20260811 10:12:28.736495 18042 raft_consensus.cc:2781] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:28.736565 18042 raft_consensus.cc:482] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:28.736590 18042 raft_consensus.cc:3037] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:28.736544 17728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4312a523e8424a2abd2b949fa20a64b4" candidate_uuid: "351d8b9947794774bb9476f59273a4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420c735f72574cc482a430759d1a611b" is_pre_election: true
I20260811 10:12:28.736678 17728 raft_consensus.cc:2454] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 351d8b9947794774bb9476f59273a4fd in term 0.
I20260811 10:12:28.796248 18042 raft_consensus.cc:504] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.796506 18042 leader_election.cc:288] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [CANDIDATE]: Term 1 election: Requested vote from peers ef9a93393d95484c99842f07d6613879 (127.8.238.3:41133), 420c735f72574cc482a430759d1a611b (127.8.238.2:42857)
I20260811 10:12:28.796769 17937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4312a523e8424a2abd2b949fa20a64b4" candidate_uuid: "351d8b9947794774bb9476f59273a4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef9a93393d95484c99842f07d6613879"
I20260811 10:12:28.796823 17728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4312a523e8424a2abd2b949fa20a64b4" candidate_uuid: "351d8b9947794774bb9476f59273a4fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "420c735f72574cc482a430759d1a611b"
I20260811 10:12:28.796911 17937 raft_consensus.cc:3037] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:28.796919 17728 raft_consensus.cc:3037] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:28.798069 17728 raft_consensus.cc:2454] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351d8b9947794774bb9476f59273a4fd in term 1.
I20260811 10:12:28.798069 17937 raft_consensus.cc:2454] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 351d8b9947794774bb9476f59273a4fd in term 1.
I20260811 10:12:28.798281 17414 leader_election.cc:302] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351d8b9947794774bb9476f59273a4fd, ef9a93393d95484c99842f07d6613879; no voters:
I20260811 10:12:28.798395 18042 raft_consensus.cc:2781] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:28.798566 18042 raft_consensus.cc:686] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [term 1 LEADER]: Becoming Leader. State: Replica: 351d8b9947794774bb9476f59273a4fd, State: Running, Role: LEADER
I20260811 10:12:28.798681 18042 consensus_queue.cc:237] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } }
I20260811 10:12:28.799908 17273 catalog_manager.cc:5495] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 351d8b9947794774bb9476f59273a4fd (127.8.238.1). New cstate: current_term: 1 leader_uuid: "351d8b9947794774bb9476f59273a4fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "351d8b9947794774bb9476f59273a4fd" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 33543 } health_report { overall_health: HEALTHY } } }
W20260811 10:12:28.813822 17813 tablet.cc:2369] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:28.821280 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.820162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.8.238.1:33543)
0811 10:12:28.820232 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:28.820235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:28.820245 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:28.820327 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:28.820328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:28.820345 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:28.820405 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:28.821055 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:28.821057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:28.821057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:28.821062 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:28.821081 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:28.821086 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:28.821140 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:28.821190 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 10:12:28.821204 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:12:28.823230 17728 raft_consensus.cc:1261] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 1 FOLLOWER]: Refusing update from remote peer 351d8b9947794774bb9476f59273a4fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:28.823257 17937 raft_consensus.cc:1261] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 1 FOLLOWER]: Refusing update from remote peer 351d8b9947794774bb9476f59273a4fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:28.823621 18046 consensus_queue.cc:1035] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "420c735f72574cc482a430759d1a611b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:28.823781 18042 consensus_queue.cc:1035] T 4312a523e8424a2abd2b949fa20a64b4 P 351d8b9947794774bb9476f59273a4fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef9a93393d95484c99842f07d6613879" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:28.826043 18051 mvcc.cc:205] Tried to move back new op lower bound from 7317271137577025536 to 7317271137480204288. Current Snapshot: MvccSnapshot[applied={T|T < 7317271137577025536}]
I20260811 10:12:28.826444 18053 mvcc.cc:205] Tried to move back new op lower bound from 7317271137577025536 to 7317271137480204288. Current Snapshot: MvccSnapshot[applied={T|T < 7317271137577025536}]
I20260811 10:12:28.829087 17276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:28.829198 17276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:28.830488 17276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75b4cd9062a64c0a8be8c08fd852a835: Sending DeleteTablet for 3 replicas of tablet 4312a523e8424a2abd2b949fa20a64b4
I20260811 10:12:28.830821 17904 tablet_service.cc:1514] Processing DeleteTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='slave'} at 127.0.0.1:52294
I20260811 10:12:28.830894 17490 tablet_service.cc:1514] Processing DeleteTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='slave'} at 127.0.0.1:52974
I20260811 10:12:28.831121 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 17381
I20260811 10:12:28.831163 17695 tablet_service.cc:1514] Processing DeleteTablet for tablet 4312a523e8424a2abd2b949fa20a64b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:28 UTC) from {username='slave'} at 127.0.0.1:39964
I20260811 10:12:28.838920 18066 tablet_replica.cc:330] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: stopping tablet replica
I20260811 10:12:28.839088 18066 raft_consensus.cc:2229] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:28.839229 18066 raft_consensus.cc:2258] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:28.845808 18066 ts_tablet_manager.cc:1905] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:28.846809 18068 tablet_replica.cc:330] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: stopping tablet replica
I20260811 10:12:28.847087 18068 raft_consensus.cc:2229] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:28.847298 18068 raft_consensus.cc:2258] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:28.847823 18068 ts_tablet_manager.cc:1905] T 4312a523e8424a2abd2b949fa20a64b4 P 420c735f72574cc482a430759d1a611b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 10:12:28.850639 17250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:33543 (error 108)
I20260811 10:12:28.850891 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 17608
I20260811 10:12:28.851210 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.851074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33543 (local address 127.0.0.1:53000)
0811 10:12:28.851127 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.851167 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:12:28.851323 17250 catalog_manager.cc:4584] TS 351d8b9947794774bb9476f59273a4fd (127.8.238.1:33543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4312a523e8424a2abd2b949fa20a64b4: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33543: connect: Connection refused (error 111)
I20260811 10:12:28.899012 18066 ts_tablet_manager.cc:1918] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:28.899160 18066 log.cc:1196] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/4312a523e8424a2abd2b949fa20a64b4
I20260811 10:12:28.899487 18066 ts_tablet_manager.cc:1939] T 4312a523e8424a2abd2b949fa20a64b4 P ef9a93393d95484c99842f07d6613879: Deleting consensus metadata
I20260811 10:12:28.900137 17252 catalog_manager.cc:4841] TS ef9a93393d95484c99842f07d6613879 (127.8.238.3:41133): tablet 4312a523e8424a2abd2b949fa20a64b4 (table test-table [id=98c0215ff36d40729940373149f36c50]) successfully deleted
I20260811 10:12:28.900913 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 17815
I20260811 10:12:28.900929 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.900776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:42857 (local address 127.0.0.1:39976)
0811 10:12:28.900842 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.900880 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.2:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 10:12:28.906944 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.906829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33543 (local address 127.0.0.1:53008)
0811 10:12:28.906873 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.906900 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 10:12:28.909461 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:28.909350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:33543 (local address 127.0.0.1:53018)
0811 10:12:28.909407 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:28.909429 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:33543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 10:12:28.919380 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 17221
2026-08-11T10:12:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (784 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 10:12:28.946671 9144 mini_oidc.cc:152] Starting JWKS server
I20260811 10:12:28.948045 9144 webserver.cc:460] Webserver started at https://127.0.0.1:40073/ using document root <none> and password file <none>
I20260811 10:12:28.948174 9144 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:12:28.948369 9144 webserver.cc:460] Webserver started at http://127.0.0.1:39089/ using document root <none> and password file <none>
2026-08-11T10:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:28Z Disabled control of system clock
I20260811 10:12:28.957841 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41125
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:37195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:41125
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40073/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/testchainca.cert with env {}
W20260811 10:12:29.042965 18080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:29.043166 18080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:29.043188 18080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:29.044307 18080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:29.044345 18080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:29.044358 18080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:29.044370 18080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:29.044382 18080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:12:29.044395 18080 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:12:29.044406 18080 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40073/jwks.json
I20260811 10:12:29.046407 18080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:41125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40073/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:29.046690 18080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:29.046931 18080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:29.051903 18090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.051913 18088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.051987 18093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:29.052297 18080 server_base.cc:1034] running on GCE node
I20260811 10:12:29.052618 18080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:29.052935 18080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:29.054113 18080 hybrid_clock.cc:648] HybridClock initialized: now 1786443149054095 us; error 45 us; skew 500 ppm
I20260811 10:12:29.055436 18080 webserver.cc:460] Webserver started at http://127.8.238.62:41091/ using document root <none> and password file <none>
I20260811 10:12:29.055713 18080 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:29.055850 18080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:29.056023 18080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:29.057499 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "8e81ec8ef37941e5ad04c98a1c4c5010"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "fa8957a1effcd24213a57eb4a6c882b8"
server_key_iv: "b15ee277ffbf29d173f2686036f7efe9"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.058024 18080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "8e81ec8ef37941e5ad04c98a1c4c5010"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "fa8957a1effcd24213a57eb4a6c882b8"
server_key_iv: "b15ee277ffbf29d173f2686036f7efe9"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.060055 18080 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 10:12:29.066025 18107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:29.066212 18080 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260811 10:12:29.066334 18080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "8e81ec8ef37941e5ad04c98a1c4c5010"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "fa8957a1effcd24213a57eb4a6c882b8"
server_key_iv: "b15ee277ffbf29d173f2686036f7efe9"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.066413 18080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:29.083318 18080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:29.083645 18080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:29.083844 18080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:29.088737 18080 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:41125
I20260811 10:12:29.088790 18187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:41125 every 8 connection(s)
I20260811 10:12:29.089172 18080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:29.090270 18191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:29.092756 18191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010: Bootstrap starting.
I20260811 10:12:29.092850 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18080
I20260811 10:12:29.093014 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:29.093248 9144 external_mini_cluster.cc:1428] Setting key d0a37d8bc5d6f868398f549e8ce2a892
I20260811 10:12:29.093526 18191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:29.093914 18191 log.cc:824] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:29.094817 18191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010: No bootstrap required, opened a new log
I20260811 10:12:29.097167 18191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } }
I20260811 10:12:29.097339 18191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:29.097374 18191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e81ec8ef37941e5ad04c98a1c4c5010, State: Initialized, Role: FOLLOWER
I20260811 10:12:29.097500 18191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } }
I20260811 10:12:29.097592 18191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:29.097617 18191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:29.097688 18191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:29.098845 18191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } }
I20260811 10:12:29.098970 18191 leader_election.cc:302] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e81ec8ef37941e5ad04c98a1c4c5010; no voters:
I20260811 10:12:29.099133 18191 leader_election.cc:288] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:29.099234 18203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:29.099392 18203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [term 1 LEADER]: Becoming Leader. State: Replica: 8e81ec8ef37941e5ad04c98a1c4c5010, State: Running, Role: LEADER
I20260811 10:12:29.099431 18191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:29.099588 18203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } }
I20260811 10:12:29.100006 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.093784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.8.238.62:41125)
0811 10:12:29.094005 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.094015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.094052 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:12:29.094752 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.094762 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.094828 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.095016 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:29.095289 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.096327 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.097513 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.098033 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.098126 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.098911 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:29.098916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:29.098916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:29.099124 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:29.099168 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:29.099182 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:29.099538 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:29.099592 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 10:12:29.099698 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260811 10:12:29.100018 18203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } } }
I20260811 10:12:29.100523 18203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.100721 18203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e81ec8ef37941e5ad04c98a1c4c5010. Latest consensus state: current_term: 1 leader_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e81ec8ef37941e5ad04c98a1c4c5010" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41125 } } }
I20260811 10:12:29.100855 18203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.101143 18225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:29.101258 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:12:29.101606 18225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:29.103958 18225 catalog_manager.cc:1347] Generated new cluster ID: bc3b3584000e4955b477295fdf0c163b
I20260811 10:12:29.104022 18225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:29.107571 18225 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:29.108146 18225 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:29.120504 18225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e81ec8ef37941e5ad04c98a1c4c5010: Generated new TSK 0
I20260811 10:12:29.120700 18225 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 10:12:29.154752 18244 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 10:12:29.154978 18244 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 10:12:29.155081 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.131650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.8.238.62:41125)
0811 10:12:29.131691 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.131693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.131866 (+ 173us) server_negotiation.cc:407] Connection header received
0811 10:12:29.131964 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.131965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.131977 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.132036 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:12:29.132223 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.132712 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.133876 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.134072 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.134123 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.134358 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:12:29.154734 (+ 20376us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 10:12:29.154993 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 10:12:29.155174 18198 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 10:12:29.155532 18243 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:29.131581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41125 (local address 127.0.0.1:34374)
0811 10:12:29.131797 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:29.131808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:29.131873 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:29.132066 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:29.132067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:29.132069 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:12:29.132153 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.132156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.132789 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.132790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.133798 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.133802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.134150 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.134152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.134221 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.134243 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:12:29.154945 (+ 20702us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 10:12:29.155071 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.238.62:41125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.238.62:41125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 10:12:29.156081 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18080
2026-08-11T10:12:29Z chronyd exiting
I20260811 10:12:29.200096 9144 test_util.cc:175] -----------------------------------------------
I20260811 10:12:29.200186 9144 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786443124069174-9144-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (254 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 10:12:29.201082 9144 mini_oidc.cc:152] Starting JWKS server
I20260811 10:12:29.201745 9144 webserver.cc:460] Webserver started at https://127.0.0.1:40705/ using document root <none> and password file <none>
I20260811 10:12:29.201784 9144 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:12:29.201890 9144 webserver.cc:460] Webserver started at http://127.0.0.1:34555/ using document root <none> and password file <none>
2026-08-11T10:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:29Z Disabled control of system clock
I20260811 10:12:29.211205 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42713
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:46245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42713
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/testchainca.cert with env {}
W20260811 10:12:29.293021 18256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:29.293218 18256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:29.293241 18256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:29.294456 18256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:29.294498 18256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:29.294512 18256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:29.294567 18256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:29.294595 18256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:12:29.294620 18256 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:12:29.294641 18256 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40705/jwks.json
I20260811 10:12:29.296191 18256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42713
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:29.296452 18256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:29.296674 18256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:29.301689 18264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.301712 18261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.301712 18262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:29.301960 18256 server_base.cc:1034] running on GCE node
I20260811 10:12:29.302220 18256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:29.302501 18256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:29.303675 18256 hybrid_clock.cc:648] HybridClock initialized: now 1786443149303653 us; error 39 us; skew 500 ppm
I20260811 10:12:29.305119 18256 webserver.cc:460] Webserver started at http://127.8.238.62:34957/ using document root <none> and password file <none>
I20260811 10:12:29.305454 18256 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:29.305538 18256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:29.305677 18256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:29.307126 18256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "845a32ece65741929bf158e24a11008e"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e965ac0506461e3b172e3f35a563ee0b"
server_key_iv: "5ed384dbec1d2eb6f4b95f9e95645a13"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.307548 18256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "845a32ece65741929bf158e24a11008e"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e965ac0506461e3b172e3f35a563ee0b"
server_key_iv: "5ed384dbec1d2eb6f4b95f9e95645a13"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.309897 18256 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 10:12:29.310842 18274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:29.311048 18256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:29.311141 18256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "845a32ece65741929bf158e24a11008e"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e965ac0506461e3b172e3f35a563ee0b"
server_key_iv: "5ed384dbec1d2eb6f4b95f9e95645a13"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.311198 18256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:29.328048 18256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:29.328308 18256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:29.328411 18256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:29.332520 18256 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:42713
I20260811 10:12:29.332576 18372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:42713 every 8 connection(s)
I20260811 10:12:29.332934 18256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:29.333951 18373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:29.335496 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18256
I20260811 10:12:29.335582 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:29.335813 9144 external_mini_cluster.cc:1428] Setting key c34f862f2c6c34113d04151f8f49c421
I20260811 10:12:29.336750 18373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e: Bootstrap starting.
I20260811 10:12:29.337502 18373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:29.337819 18373 log.cc:824] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:29.338574 18373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e: No bootstrap required, opened a new log
I20260811 10:12:29.341044 18373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } }
I20260811 10:12:29.341244 18373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:29.341307 18373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845a32ece65741929bf158e24a11008e, State: Initialized, Role: FOLLOWER
I20260811 10:12:29.341447 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.336323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55060 (local address 127.8.238.62:42713)
0811 10:12:29.336535 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.336542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.336566 (+ 24us) server_negotiation.cc:407] Connection header received
0811 10:12:29.337016 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.337024 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.337082 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.337197 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:29.337450 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.338404 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.339489 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.339948 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.340040 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.340735 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:29.340739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:29.340739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:29.340894 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:29.340928 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:29.340938 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:29.340988 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:29.341098 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 10:12:29.341185 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260811 10:12:29.341456 18373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } }
I20260811 10:12:29.341581 18373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:29.341621 18373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:29.341667 18373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:29.342509 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:12:29.407485 18373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } }
I20260811 10:12:29.407706 18373 leader_election.cc:302] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 845a32ece65741929bf158e24a11008e; no voters:
I20260811 10:12:29.408022 18373 leader_election.cc:288] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:29.408087 18380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:29.408308 18380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [term 1 LEADER]: Becoming Leader. State: Replica: 845a32ece65741929bf158e24a11008e, State: Running, Role: LEADER
I20260811 10:12:29.408411 18373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:29.408440 18380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } }
I20260811 10:12:29.408802 18380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "845a32ece65741929bf158e24a11008e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } } }
I20260811 10:12:29.408890 18380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.409037 18380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 845a32ece65741929bf158e24a11008e. Latest consensus state: current_term: 1 leader_uuid: "845a32ece65741929bf158e24a11008e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "845a32ece65741929bf158e24a11008e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42713 } } }
I20260811 10:12:29.409113 18380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.410077 18386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:29.410630 18386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:29.412211 18386 catalog_manager.cc:1347] Generated new cluster ID: 976b87dc315046fdb68ece24b57f2356
I20260811 10:12:29.412267 18386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:29.419945 18386 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:29.420457 18386 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:29.425020 18386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 845a32ece65741929bf158e24a11008e: Generated new TSK 0
I20260811 10:12:29.425200 18386 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 10:12:29.452047 18417 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 10:12:29.452134 18417 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 10:12:29.452472 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.438047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.8.238.62:42713)
0811 10:12:29.438127 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.438129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.438216 (+ 87us) server_negotiation.cc:407] Connection header received
0811 10:12:29.438263 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.438264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.438277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.438356 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:12:29.438487 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.439302 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.440134 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.440459 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.440534 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.440683 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:12:29.452241 (+ 11558us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 10:12:29.452393 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 10:12:29.452530 18376 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55072: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 10:12:29.452517 18416 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:29.437971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42713 (local address 127.0.0.1:55072)
0811 10:12:29.438133 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:29.438144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:29.438208 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:29.438361 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:29.438362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:29.438364 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:12:29.438420 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.438421 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.439383 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.439383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.440049 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.440052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.440541 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.440542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.440582 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.440585 (+ 3us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:12:29.452343 (+ 11758us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 10:12:29.452438 (+ 95us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.238.62:42713: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40705/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260811 10:12:29.452847 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18256
2026-08-11T10:12:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 10:12:29.501160 9144 mini_oidc.cc:152] Starting JWKS server
I20260811 10:12:29.501888 9144 webserver.cc:460] Webserver started at https://127.0.0.1:35861/ using document root <none> and password file <none>
I20260811 10:12:29.501931 9144 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 10:12:29.502130 9144 webserver.cc:460] Webserver started at http://127.0.0.1:37351/ using document root <none> and password file <none>
2026-08-11T10:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:29Z Disabled control of system clock
I20260811 10:12:29.511982 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40789
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:40789
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35861/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 10:12:29.596171 18427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:29.596377 18427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:29.596400 18427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:29.597503 18427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:29.597537 18427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:29.597549 18427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:29.597563 18427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:29.597574 18427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:12:29.597586 18427 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:12:29.597599 18427 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35861/jwks.json
I20260811 10:12:29.598974 18427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:40789
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35861/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:29.599166 18427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:29.599390 18427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:29.604300 18436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.604327 18437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.604341 18440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:29.604722 18427 server_base.cc:1034] running on GCE node
I20260811 10:12:29.605032 18427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:29.605363 18427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:29.606560 18427 hybrid_clock.cc:648] HybridClock initialized: now 1786443149606534 us; error 37 us; skew 500 ppm
I20260811 10:12:29.608130 18427 webserver.cc:460] Webserver started at http://127.8.238.62:36217/ using document root <none> and password file <none>
I20260811 10:12:29.608394 18427 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:29.608472 18427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:29.608636 18427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:29.609894 18427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "e22b527c48ba4c80aa620c393c86c2c4"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "c854aa70a67dc090441287ad777e617c"
server_key_iv: "df1d61e06fca0c1bd5ff86566612d115"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.610312 18427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "e22b527c48ba4c80aa620c393c86c2c4"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "c854aa70a67dc090441287ad777e617c"
server_key_iv: "df1d61e06fca0c1bd5ff86566612d115"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.612046 18427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 10:12:29.615598 18455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:29.615862 18427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 10:12:29.615986 18427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e22b527c48ba4c80aa620c393c86c2c4"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "c854aa70a67dc090441287ad777e617c"
server_key_iv: "df1d61e06fca0c1bd5ff86566612d115"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.616080 18427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:29.624665 18427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:29.624974 18427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:29.625125 18427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:29.629577 18427 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:40789
I20260811 10:12:29.629603 18536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:40789 every 8 connection(s)
I20260811 10:12:29.629961 18427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:29.630676 18537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:29.633628 18537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4: Bootstrap starting.
I20260811 10:12:29.634259 18537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:29.634579 18537 log.cc:824] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:29.635288 18537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4: No bootstrap required, opened a new log
I20260811 10:12:29.636976 18537 raft_consensus.cc:348] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } }
I20260811 10:12:29.637115 18537 raft_consensus.cc:374] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:29.637138 18537 raft_consensus.cc:729] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e22b527c48ba4c80aa620c393c86c2c4, State: Initialized, Role: FOLLOWER
I20260811 10:12:29.637326 18537 consensus_queue.cc:260] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } }
I20260811 10:12:29.637408 18537 raft_consensus.cc:388] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:29.637451 18537 raft_consensus.cc:482] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:29.637509 18537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:29.638537 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18427
I20260811 10:12:29.638648 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:29.638842 9144 external_mini_cluster.cc:1428] Setting key e27e805a8c57eaba6e38ad875d544b56
I20260811 10:12:29.643617 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.639333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.8.238.62:40789)
0811 10:12:29.639564 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.639571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.639596 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:12:29.639686 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.639692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.639787 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.639905 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:29.640072 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.641237 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.641994 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.642376 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.642467 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.643085 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:29.643088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:29.643089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:29.643247 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:29.643281 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:29.643291 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:29.643342 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:29.643398 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 10:12:29.643455 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260811 10:12:29.644544 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:12:29.712186 18537 raft_consensus.cc:504] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } }
I20260811 10:12:29.712423 18537 leader_election.cc:302] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e22b527c48ba4c80aa620c393c86c2c4; no voters:
I20260811 10:12:29.712736 18537 leader_election.cc:288] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:29.712824 18546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:29.713035 18546 raft_consensus.cc:686] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [term 1 LEADER]: Becoming Leader. State: Replica: e22b527c48ba4c80aa620c393c86c2c4, State: Running, Role: LEADER
I20260811 10:12:29.713050 18537 sys_catalog.cc:564] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:29.713146 18546 consensus_queue.cc:237] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } }
I20260811 10:12:29.713611 18548 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e22b527c48ba4c80aa620c393c86c2c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } } }
I20260811 10:12:29.713703 18548 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.713938 18551 sys_catalog.cc:455] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e22b527c48ba4c80aa620c393c86c2c4. Latest consensus state: current_term: 1 leader_uuid: "e22b527c48ba4c80aa620c393c86c2c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e22b527c48ba4c80aa620c393c86c2c4" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40789 } } }
I20260811 10:12:29.714068 18551 sys_catalog.cc:458] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:29.714347 18559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:29.714937 18559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:29.718811 18559 catalog_manager.cc:1347] Generated new cluster ID: a696dfb488d646b4b40ce90a9b7cda4e
I20260811 10:12:29.718880 18559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:29.736088 18559 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:29.736613 18559 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:29.742165 18559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e22b527c48ba4c80aa620c393c86c2c4: Generated new TSK 0
I20260811 10:12:29.742360 18559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:12:29.754204 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.750567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43264 (local address 127.8.238.62:40789)
0811 10:12:29.750618 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.750620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.750683 (+ 63us) server_negotiation.cc:407] Connection header received
0811 10:12:29.750768 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.750768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.750778 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.750836 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:12:29.750977 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.751478 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.752643 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.752832 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.752885 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.753085 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:12:29.754008 (+ 923us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 10:12:29.754139 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 10:12:29.754263 18543 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 10:12:29.754585 18581 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:29.750472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40789 (local address 127.0.0.1:43264)
0811 10:12:29.750627 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:29.750638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:29.750701 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:29.750850 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:29.750852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:29.750854 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:12:29.750919 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.750922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.751558 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.751559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.752563 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:29.752567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.752913 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.752915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:29.752984 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.752990 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:12:29.754114 (+ 1124us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 10:12:29.754215 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.238.62:40789: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260811 10:12:29.754863 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18427
E20260811 10:12:29.771157 18582 webserver.cc:533] Webserver: sslize failed:
E20260811 10:12:29.771250 18582 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T10:12:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (304 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T10:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:29Z Disabled control of system clock
I20260811 10:12:29.814797 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35213
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:40125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35213
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/testchainca.cert with env {}
W20260811 10:12:29.902150 18592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:29.902376 18592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:29.902415 18592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:29.904299 18592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:29.904364 18592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:29.904388 18592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:29.904417 18592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:29.904472 18592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 10:12:29.904505 18592 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 10:12:29.904531 18592 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 10:12:29.906095 18592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:29.906363 18592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:29.906603 18592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:29.911422 18612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.911463 18602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:29.911422 18605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:29.911646 18592 server_base.cc:1034] running on GCE node
I20260811 10:12:29.911933 18592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:29.912258 18592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:29.913429 18592 hybrid_clock.cc:648] HybridClock initialized: now 1786443149913408 us; error 57 us; skew 500 ppm
I20260811 10:12:29.914661 18592 webserver.cc:460] Webserver started at http://127.8.238.62:46393/ using document root <none> and password file <none>
I20260811 10:12:29.914930 18592 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:29.915010 18592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:29.915159 18592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:29.916254 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "e30e3019ec7f4d2fb09a725dffc15f45"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e660d82df278701fe1052c2c384211e0"
server_key_iv: "25f31f494c75fcbdac9ec1eeb16eed2f"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.916621 18592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "e30e3019ec7f4d2fb09a725dffc15f45"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e660d82df278701fe1052c2c384211e0"
server_key_iv: "25f31f494c75fcbdac9ec1eeb16eed2f"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.917910 18592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:12:29.918691 18621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:29.918893 18592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:29.918996 18592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e30e3019ec7f4d2fb09a725dffc15f45"
format_stamp: "Formatted at 2026-08-11 10:12:29 on dist-test-slave-7f01"
server_key: "e660d82df278701fe1052c2c384211e0"
server_key_iv: "25f31f494c75fcbdac9ec1eeb16eed2f"
server_key_version: "encryptionkey@0"
I20260811 10:12:29.919072 18592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:29.934297 18592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:29.934605 18592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:29.934752 18592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:29.939535 18592 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35213
I20260811 10:12:29.939582 18718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35213 every 8 connection(s)
I20260811 10:12:29.939929 18592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:29.940891 18720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:29.943101 18720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45: Bootstrap starting.
I20260811 10:12:29.943650 18720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:29.943979 18720 log.cc:824] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:29.944710 18720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45: No bootstrap required, opened a new log
I20260811 10:12:29.946264 18720 raft_consensus.cc:348] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } }
I20260811 10:12:29.946396 18720 raft_consensus.cc:374] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:29.946419 18720 raft_consensus.cc:729] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e30e3019ec7f4d2fb09a725dffc15f45, State: Initialized, Role: FOLLOWER
I20260811 10:12:29.946552 18720 consensus_queue.cc:260] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } }
I20260811 10:12:29.946630 18720 raft_consensus.cc:388] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:29.946663 18720 raft_consensus.cc:482] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:29.946705 18720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:29.949509 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18592
I20260811 10:12:29.949632 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:29.949819 9144 external_mini_cluster.cc:1428] Setting key cc4af207d8525a35cb2f060612683bca
I20260811 10:12:29.954468 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:29.950327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52500 (local address 127.8.238.62:35213)
0811 10:12:29.950545 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:29.950552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:29.950577 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:12:29.950671 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:29.950677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:29.950726 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:29.950835 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:29.951044 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.951966 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.952694 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:29.953081 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:29.953166 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:29.953745 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:29.953749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:29.953749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:12:29.953918 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:29.953950 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:12:29.953960 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:29.954011 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:29.954077 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 10:12:29.954165 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260811 10:12:29.955406 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:12:30.020278 18720 raft_consensus.cc:504] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } }
I20260811 10:12:30.020522 18720 leader_election.cc:302] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e30e3019ec7f4d2fb09a725dffc15f45; no voters:
I20260811 10:12:30.020813 18720 leader_election.cc:288] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:30.020906 18726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:30.021095 18726 raft_consensus.cc:686] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [term 1 LEADER]: Becoming Leader. State: Replica: e30e3019ec7f4d2fb09a725dffc15f45, State: Running, Role: LEADER
I20260811 10:12:30.021122 18720 sys_catalog.cc:564] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:30.021205 18726 consensus_queue.cc:237] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } }
I20260811 10:12:30.021656 18727 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } } }
I20260811 10:12:30.021763 18727 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:30.022049 18729 sys_catalog.cc:455] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e30e3019ec7f4d2fb09a725dffc15f45. Latest consensus state: current_term: 1 leader_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e30e3019ec7f4d2fb09a725dffc15f45" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35213 } } }
I20260811 10:12:30.022439 18729 sys_catalog.cc:458] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:30.023033 18741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:30.023504 18741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:30.025236 18741 catalog_manager.cc:1347] Generated new cluster ID: 2de0cac307364ec591e9840a2b9bcc45
I20260811 10:12:30.025301 18741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:30.040712 18741 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:30.041393 18741 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:30.056885 18741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e30e3019ec7f4d2fb09a725dffc15f45: Generated new TSK 0
I20260811 10:12:30.057081 18741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:12:30.133579 18725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:30.073642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.8.238.62:35213)
0811 10:12:30.073705 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:30.073707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:30.073789 (+ 82us) server_negotiation.cc:407] Connection header received
0811 10:12:30.073833 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:30.073834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:30.073848 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:30.073920 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0811 10:12:30.074038 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:30.074799 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:30.075593 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:30.075912 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:30.075977 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:30.076148 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 10:12:30.133264 (+ 57116us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 10:12:30.133480 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 10:12:30.133668 18725 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52504: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 10:12:30.133683 18763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:30.073560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35213 (local address 127.0.0.1:52504)
0811 10:12:30.073725 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:30.073733 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:30.073783 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:30.073923 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:30.073924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:30.073926 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 10:12:30.073979 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:30.073981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:30.074909 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:30.074911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:30.075507 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:30.075509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:30.076002 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:30.076003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:30.076050 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:30.076054 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 10:12:30.133439 (+ 57385us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 10:12:30.133572 (+ 133us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.238.62:35213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260811 10:12:30.134112 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18592
2026-08-11T10:12:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (352 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:30 dist-test-slave-7f01 krb5kdc[18777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:30 dist-test-slave-7f01 krb5kdc[18777](info): set up 2 sockets
Aug 11 10:12:30 dist-test-slave-7f01 krb5kdc[18777](info): commencing operation
krb5kdc: starting...
W20260811 10:12:31.800333 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.449s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:31 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:31.924616 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:37745
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:37745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:32.007383 18803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:32.007628 18803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:32.007665 18803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:32.008821 18803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:32.008900 18803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:32.008924 18803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:32.008944 18803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:32.009009 18803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:32.010444 18803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:37745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:37745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:32.010722 18803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:32.010958 18803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:32.016264 18815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.016297 18816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.016290 18818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:32.016634 18803 server_base.cc:1034] running on GCE node
I20260811 10:12:32.016945 18803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:32.017297 18803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:32.018486 18803 hybrid_clock.cc:648] HybridClock initialized: now 1786443152018461 us; error 34 us; skew 500 ppm
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.024842 18803 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:32.025305 18803 webserver.cc:460] Webserver started at http://127.8.238.62:43371/ using document root <none> and password file <none>
I20260811 10:12:32.025561 18803 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:32.025632 18803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:32.025781 18803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:32.026916 18803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "a1510d60661944f08127c33ae0c3a5c7"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "76f1240b469cb72f531595f6f79129df"
server_key_iv: "d0f26ba4003550249b11314bac4bf667"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.027307 18803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "a1510d60661944f08127c33ae0c3a5c7"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "76f1240b469cb72f531595f6f79129df"
server_key_iv: "d0f26ba4003550249b11314bac4bf667"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.028750 18803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:32.029524 18833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.029729 18803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:32.029870 18803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "a1510d60661944f08127c33ae0c3a5c7"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "76f1240b469cb72f531595f6f79129df"
server_key_iv: "d0f26ba4003550249b11314bac4bf667"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.029953 18803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:32.051414 18803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:32.053768 18803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:32.053934 18803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:32.058019 18803 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:37745
I20260811 10:12:32.058066 18929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:37745 every 8 connection(s)
I20260811 10:12:32.058445 18803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:32.059336 18930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:32.061445 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18803
I20260811 10:12:32.061550 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:32.061759 9144 external_mini_cluster.cc:1428] Setting key 5cdb0e216cb69d05793fbfdcddbb03f5
I20260811 10:12:32.062093 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7: Bootstrap starting.
I20260811 10:12:32.062809 18930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:32.063118 18930 log.cc:824] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:32.063920 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7: No bootstrap required, opened a new log
I20260811 10:12:32.066102 18930 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } }
I20260811 10:12:32.066268 18930 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:32.066294 18930 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1510d60661944f08127c33ae0c3a5c7, State: Initialized, Role: FOLLOWER
I20260811 10:12:32.066431 18930 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } }
I20260811 10:12:32.066499 18930 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:32.066521 18930 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:32.066583 18930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:32.067335 18930 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } }
I20260811 10:12:32.067463 18930 leader_election.cc:302] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1510d60661944f08127c33ae0c3a5c7; no voters:
I20260811 10:12:32.067644 18930 leader_election.cc:288] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:32.067713 18940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:32.067948 18930 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:32.067986 18940 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [term 1 LEADER]: Becoming Leader. State: Replica: a1510d60661944f08127c33ae0c3a5c7, State: Running, Role: LEADER
I20260811 10:12:32.068089 18940 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } }
I20260811 10:12:32.068403 18940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1510d60661944f08127c33ae0c3a5c7. Latest consensus state: current_term: 1 leader_uuid: "a1510d60661944f08127c33ae0c3a5c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } } }
I20260811 10:12:32.068495 18940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:32.068414 18942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1510d60661944f08127c33ae0c3a5c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1510d60661944f08127c33ae0c3a5c7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37745 } } }
I20260811 10:12:32.068619 18942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:32.068782 18949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:32.069291 18949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:32.072988 18949 catalog_manager.cc:1347] Generated new cluster ID: dfd135a36ec54c88b5cae4af8cef98d9
I20260811 10:12:32.073050 18949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:32.074810 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.062275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.8.238.62:37745)
0811 10:12:32.062471 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.062478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.062504 (+ 26us) server_negotiation.cc:407] Connection header received
0811 10:12:32.063010 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.063019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.063067 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.063181 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.064061 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.065024 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.066154 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.066790 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.066876 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.069902 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.069907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.069907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.069921 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.073277 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.073667 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.073670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.073671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.073768 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.073986 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.073987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.073988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.074229 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.074417 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.074489 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.074623 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260811 10:12:32.092010 18949 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:32.092777 18949 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:32.099809 18949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1510d60661944f08127c33ae0c3a5c7: Generated new TSK 0
I20260811 10:12:32.100013 18949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:32.134196 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:32.218184 18967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:32.218384 18967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:32.218407 18967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:32.219551 18967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:32.219594 18967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:32.219619 18967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:32.221101 18967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:32.221365 18967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:32.221601 18967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:32.226651 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.226691 18977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:32.226799 18967 server_base.cc:1034] running on GCE node
W20260811 10:12:32.226675 18975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:32.227063 18967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:32.227306 18967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:32.228480 18967 hybrid_clock.cc:648] HybridClock initialized: now 1786443152228465 us; error 46 us; skew 500 ppm
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.230938 18967 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:32.231366 18967 webserver.cc:460] Webserver started at http://127.8.238.1:33055/ using document root <none> and password file <none>
I20260811 10:12:32.231611 18967 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:32.231683 18967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:32.231854 18967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:32.232901 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "0b4b06963851483c8fb6a95f89061a2e"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "a5b7a9e1e0184a97f6a60b0db355120e"
server_key_iv: "9dcebff3598ffa19398b3421507f0f20"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.233289 18967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "0b4b06963851483c8fb6a95f89061a2e"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "a5b7a9e1e0184a97f6a60b0db355120e"
server_key_iv: "9dcebff3598ffa19398b3421507f0f20"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.234651 18967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:32.235389 18999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.235597 18967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:32.235710 18967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "0b4b06963851483c8fb6a95f89061a2e"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "a5b7a9e1e0184a97f6a60b0db355120e"
server_key_iv: "9dcebff3598ffa19398b3421507f0f20"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.235852 18967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:32.249392 18967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:32.251202 18967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:32.251371 18967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:32.251621 18967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:32.252012 18967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:32.252074 18967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.252110 18967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:32.252156 18967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.258433 18967 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:41417
I20260811 10:12:32.258502 19195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:41417 every 8 connection(s)
I20260811 10:12:32.258807 18967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:32.259428 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 18967
I20260811 10:12:32.259588 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:32.259853 9144 external_mini_cluster.cc:1428] Setting key 8f9d83cbca3260bddc8c2127997f3824
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
I20260811 10:12:32.270143 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.259589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:49851 (local address 127.8.238.62:37745)
0811 10:12:32.259641 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.259643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.259895 (+ 252us) server_negotiation.cc:407] Connection header received
0811 10:12:32.261263 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.261265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.261275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.261324 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.262196 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.262735 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.264270 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.264463 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.264518 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.267024 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.267025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.267025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.267034 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.268574 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.269119 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.269120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.269121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.269195 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.269466 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.269467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.269468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.269588 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.269661 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.269951 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.270051 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:32.270310 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.259546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37745 (local address 127.8.238.1:49851)
0811 10:12:32.259758 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:32.259810 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:32.261159 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:32.261403 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:32.261407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:32.261808 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:32.262105 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.262113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.262817 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.262818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.264190 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.264194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.264550 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.264551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.264702 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.265224 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:32.265234 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:32.266919 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:32.268680 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.268688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.268691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.269005 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.269274 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.269274 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.269275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.269390 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.269665 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:32.269674 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:32.269813 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 10:12:32.269915 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 10:12:32.270076 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260811 10:12:32.270851 19196 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37745
I20260811 10:12:32.270939 19196 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:32.271163 19196 heartbeater.cc:511] Master 127.8.238.62:37745 requested a full tablet report, sending...
I20260811 10:12:32.271703 18860 ts_manager.cc:194] Registered new tserver with Master: 0b4b06963851483c8fb6a95f89061a2e (127.8.238.1:41417)
I20260811 10:12:32.272338 18860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:49851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:32.288190 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:32.373317 19205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:32.373530 19205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:32.373554 19205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:32.374650 19205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:32.374684 19205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:32.374709 19205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:32.376174 19205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:32.376359 19205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:32.376564 19205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:32.381598 19211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.381610 19215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.381623 19217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:32.381816 19205 server_base.cc:1034] running on GCE node
I20260811 10:12:32.382121 19205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:32.382372 19205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:32.383556 19205 hybrid_clock.cc:648] HybridClock initialized: now 1786443152383539 us; error 46 us; skew 500 ppm
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.386283 19205 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:32.386673 19205 webserver.cc:460] Webserver started at http://127.8.238.2:35603/ using document root <none> and password file <none>
I20260811 10:12:32.386871 19205 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:32.386914 19205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:32.387002 19205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:32.388090 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "094e3cb74291478db55f4f9c6dc226e9"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "fd189ff8763b9d548bab4e750b9aadb1"
server_key_iv: "3960d9afe11283a2215110d2edfd191b"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.388417 19205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "094e3cb74291478db55f4f9c6dc226e9"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "fd189ff8763b9d548bab4e750b9aadb1"
server_key_iv: "3960d9afe11283a2215110d2edfd191b"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.389696 19205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:12:32.390375 19230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.390552 19205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.390663 19205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "094e3cb74291478db55f4f9c6dc226e9"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "fd189ff8763b9d548bab4e750b9aadb1"
server_key_iv: "3960d9afe11283a2215110d2edfd191b"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.390741 19205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:32.401443 19205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:32.403906 19205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:32.404068 19205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:32.404306 19205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:32.404632 19205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:32.404687 19205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.404729 19205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:32.404754 19205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.410789 19205 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:35527
I20260811 10:12:32.410869 19425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:35527 every 8 connection(s)
I20260811 10:12:32.411194 19205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:32.413671 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 19205
I20260811 10:12:32.413784 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:32.413975 9144 external_mini_cluster.cc:1428] Setting key d732b5d25c11b77ea181645f21b0879b
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:32.421478 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.412016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:48997 (local address 127.8.238.62:37745)
0811 10:12:32.412065 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.412068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.412292 (+ 224us) server_negotiation.cc:407] Connection header received
0811 10:12:32.413284 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.413286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.413301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.413354 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.414073 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.414746 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.415772 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.415988 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.416071 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.418507 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.418508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.418509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.418517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.420033 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.420468 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.420470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.420470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.420537 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.420803 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.420804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.420804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.420922 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.421006 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.421292 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.421385 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:12:32.421726 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.411973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37745 (local address 127.8.238.2:48997)
0811 10:12:32.412163 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:32.412199 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:32.413156 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:32.413421 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:32.413425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:32.413712 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:32.413976 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.413981 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.414839 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.414840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.415653 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.415656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.416079 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.416080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.416221 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.416756 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:32.416767 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:32.418406 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:32.420139 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.420146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.420149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.420388 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.420613 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.420614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.420614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.420726 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.420997 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:32.421004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:32.421152 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 10:12:32.421265 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 10:12:32.421426 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260811 10:12:32.422289 19426 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37745
I20260811 10:12:32.422408 19426 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:32.422609 19426 heartbeater.cc:511] Master 127.8.238.62:37745 requested a full tablet report, sending...
I20260811 10:12:32.423071 18860 ts_manager.cc:194] Registered new tserver with Master: 094e3cb74291478db55f4f9c6dc226e9 (127.8.238.2:35527)
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
I20260811 10:12:32.423626 18860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:48997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:32.442298 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:32.524879 19437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:32.525136 19437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:32.525178 19437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:32.526331 19437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:32.526402 19437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:32.526446 19437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:32.527989 19437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:32.528265 19437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:32.528499 19437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:32.533650 19453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.533676 19446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:32.533684 19449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:32.534039 19437 server_base.cc:1034] running on GCE node
I20260811 10:12:32.534281 19437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:32.534519 19437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:32.535678 19437 hybrid_clock.cc:648] HybridClock initialized: now 1786443152535669 us; error 27 us; skew 500 ppm
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.538167 19437 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:32.538676 19437 webserver.cc:460] Webserver started at http://127.8.238.3:41221/ using document root <none> and password file <none>
I20260811 10:12:32.538928 19437 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:32.538975 19437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:32.539064 19437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:32.540233 19437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "4798190eec694509b60aaebf317cc050"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "58434f5683eae0e0b1c6f22802fd9e7b"
server_key_iv: "332bdfac2bdcf36da16dbf77f03f008d"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.540544 19437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "4798190eec694509b60aaebf317cc050"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "58434f5683eae0e0b1c6f22802fd9e7b"
server_key_iv: "332bdfac2bdcf36da16dbf77f03f008d"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.541778 19437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:32.542455 19468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.542640 19437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:32.542732 19437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "4798190eec694509b60aaebf317cc050"
format_stamp: "Formatted at 2026-08-11 10:12:32 on dist-test-slave-7f01"
server_key: "58434f5683eae0e0b1c6f22802fd9e7b"
server_key_iv: "332bdfac2bdcf36da16dbf77f03f008d"
server_key_version: "encryptionkey@0"
I20260811 10:12:32.542786 19437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:32.557726 19437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:32.560074 19437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:32.560184 19437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:32.560364 19437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:32.560693 19437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:32.560729 19437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.560750 19437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:32.560798 19437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:32.566672 19437 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:42985
I20260811 10:12:32.566761 19656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:42985 every 8 connection(s)
I20260811 10:12:32.567083 19437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:32.567742 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 19437
I20260811 10:12:32.567889 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:32.568117 9144 external_mini_cluster.cc:1428] Setting key 7269657ca9c0caca9becd80228d7b451
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:32.576761 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.567915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:37667 (local address 127.8.238.62:37745)
0811 10:12:32.567970 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.567973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.568219 (+ 246us) server_negotiation.cc:407] Connection header received
0811 10:12:32.569549 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.569550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.569562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.569610 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.570306 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.570806 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.571823 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.572041 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.572101 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.574190 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.574191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.574192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.574198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.575600 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.576009 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.576010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.576010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.576063 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.576263 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.576263 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.576263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.576351 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.576399 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.576631 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.576695 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:12:32.576953 19659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.567888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37745 (local address 127.8.238.3:37667)
0811 10:12:32.568075 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:32.568124 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:32.569427 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:32.569682 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:32.569686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:32.569972 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:32.570169 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.570174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.570893 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.570894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.571708 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:32.571710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.572121 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.572122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:32.572222 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.572649 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:32.572656 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:32.574105 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:32.575674 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.575680 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.575682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.575943 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.576125 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:32.576126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:32.576126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:32.576223 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:32.576401 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:32.576405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:32.576522 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 10:12:32.576603 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:12:32.576719 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260811 10:12:32.577499 19657 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37745
I20260811 10:12:32.577606 19657 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:32.577782 19657 heartbeater.cc:511] Master 127.8.238.62:37745 requested a full tablet report, sending...
I20260811 10:12:32.578223 18860 ts_manager.cc:194] Registered new tserver with Master: 4798190eec694509b60aaebf317cc050 (127.8.238.3:42985)
I20260811 10:12:32.578558 18860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:37667
I20260811 10:12:32.579828 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/test-admin.keytab.
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.596760 9144 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 10:12:32.598526 19668 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:32.597704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.8.238.1:41417)
0811 10:12:32.597886 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.597892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.597907 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:12:32.597934 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.597936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.597952 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.598033 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.598440 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57784: BlockingRecv error: recv got EOF from 127.0.0.1:57784 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
W20260811 10:12:32.598590 9144 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc' of type FILE
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.599524 9144 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.600476 9144 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:32.607848 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.600982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.8.238.1:41417)
0811 10:12:32.601038 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.601040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.601064 (+ 24us) server_negotiation.cc:407] Connection header received
0811 10:12:32.601106 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.601107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.601116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.601158 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.601535 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.602301 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.603000 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.603309 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.603359 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.605393 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.605403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.605403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.605410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.606814 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.607135 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.607137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.607138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.607196 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.607361 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.607362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.607362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.607486 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.607595 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.607651 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.607713 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 10:12:32.609692 19668 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:32.609114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57800 (local address 127.8.238.1:41417)
0811 10:12:32.609148 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.609150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.609164 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:12:32.609185 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.609185 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.609194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.609240 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.609632 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57800: BlockingRecv error: recv got EOF from 127.0.0.1:57800 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260811 10:12:32.609786 9144 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc' of type FILE
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.610699 9144 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.611549 9144 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:32.618078 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.612100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57810 (local address 127.8.238.1:41417)
0811 10:12:32.612155 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.612156 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.612183 (+ 27us) server_negotiation.cc:407] Connection header received
0811 10:12:32.612247 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.612248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.612256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.612301 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.612648 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.613130 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.613746 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.613939 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.613988 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.615952 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.615954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.615954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.615960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.617089 (+ 1129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.617385 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.617386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.617386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.617470 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.617640 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.617641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.617641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.617726 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.617792 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.617875 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.617943 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 10:12:32.619402 19668 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:32.618975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.8.238.1:41417)
0811 10:12:32.619008 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.619009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.619040 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:12:32.619075 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.619076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.619082 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.619127 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.619358 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57812: BlockingRecv error: recv got EOF from 127.0.0.1:57812 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260811 10:12:32.619517 9144 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786443124069174-9144-0/krb5kdc/krb5cc' of type FILE
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.620433 9144 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:32.621246 9144 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[18777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:12:32.627480 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:32.621688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.8.238.1:41417)
0811 10:12:32.621722 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:32.621723 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:32.621769 (+ 46us) server_negotiation.cc:407] Connection header received
0811 10:12:32.621817 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:32.621818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:32.621824 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:32.621871 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:32.622213 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.622647 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.623268 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:32.623466 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:32.623518 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:32.625477 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:32.625479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:32.625479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:32.625485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:32.626606 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.626924 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.626925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.626925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.626975 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:32.627139 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:32.627140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:32.627140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:32.627218 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:32.627283 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:32.627379 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 10:12:32.627430 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 10:12:32.628166 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18967
I20260811 10:12:32.638374 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 19205
I20260811 10:12:32.658912 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 19437
I20260811 10:12:32.670413 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 18803
2026-08-11T10:12:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2346 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[19694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[19694](info): set up 2 sockets
Aug 11 10:12:32 dist-test-slave-7f01 krb5kdc[19694](info): commencing operation
krb5kdc: starting...
W20260811 10:12:34.029872 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:34Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab.
Entry for principal oryx/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab.
I20260811 10:12:34.156908 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43017
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab
--principal=oryx/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:43017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:34.240573 19712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:34.240787 19712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:34.240809 19712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:34.242081 19712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:34.242126 19712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:34.242141 19712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:34.242154 19712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:34.242205 19712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:34.243688 19712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:43017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab
--principal=oryx/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:34.243986 19712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:34.244248 19712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:34.249056 19719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.249060 19731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.249138 19720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.249353 19712 server_base.cc:1034] running on GCE node
I20260811 10:12:34.249679 19712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:34.250003 19712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:34.251168 19712 hybrid_clock.cc:648] HybridClock initialized: now 1786443154251157 us; error 31 us; skew 500 ppm
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:34.253726 19712 init.cc:377] Logged in from keytab as oryx/127.8.238.62@KRBTEST.COM (short username oryx)
I20260811 10:12:34.254145 19712 webserver.cc:460] Webserver started at http://127.8.238.62:32913/ using document root <none> and password file <none>
I20260811 10:12:34.254376 19712 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:34.254446 19712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:34.254598 19712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:34.255625 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "63a5cd6331965c48e3320297973b9d30"
server_key_iv: "085f3cfd46fc2aeb6822b47e54d1acbf"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.256054 19712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "63a5cd6331965c48e3320297973b9d30"
server_key_iv: "085f3cfd46fc2aeb6822b47e54d1acbf"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.257431 19712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:34.258214 19738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.258419 19712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:34.258548 19712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "63a5cd6331965c48e3320297973b9d30"
server_key_iv: "085f3cfd46fc2aeb6822b47e54d1acbf"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.258628 19712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:34.269220 19712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:34.271196 19712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:34.271368 19712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:34.275630 19712 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:43017
I20260811 10:12:34.275677 19824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:43017 every 8 connection(s)
I20260811 10:12:34.276057 19712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:34.277145 19825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:34.279538 19825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Bootstrap starting.
I20260811 10:12:34.280165 19825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:34.280485 19825 log.cc:824] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:34.281307 19825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: No bootstrap required, opened a new log
I20260811 10:12:34.282095 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 19712
I20260811 10:12:34.282217 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:34.282435 9144 external_mini_cluster.cc:1428] Setting key 498fe7491bbc7662c91828bdbd11b71a
I20260811 10:12:34.283602 19825 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } }
I20260811 10:12:34.283790 19825 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:34.283857 19825 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c28fcf12f4425d919b8ff54d0fd0b3, State: Initialized, Role: FOLLOWER
I20260811 10:12:34.284003 19825 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } }
I20260811 10:12:34.284097 19825 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:34.284132 19825 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:34.284184 19825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:34.285166 19825 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } }
I20260811 10:12:34.285301 19825 leader_election.cc:302] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6c28fcf12f4425d919b8ff54d0fd0b3; no voters:
I20260811 10:12:34.285488 19825 leader_election.cc:288] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:34.285513 19835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:34.285631 19835 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 LEADER]: Becoming Leader. State: Replica: b6c28fcf12f4425d919b8ff54d0fd0b3, State: Running, Role: LEADER
I20260811 10:12:34.285717 19835 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } }
I20260811 10:12:34.285758 19825 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:34.286039 19835 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6c28fcf12f4425d919b8ff54d0fd0b3. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } }
I20260811 10:12:34.286195 19835 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:34.286055 19836 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } }
I20260811 10:12:34.286377 19836 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:34.287125 19843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:34.288159 19843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:34.289909 19843 catalog_manager.cc:1347] Generated new cluster ID: 925ee9ecc2c34d71bcb6c262c9cc4df7
I20260811 10:12:34.289968 19843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:34.293637 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.282967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.8.238.62:43017)
0811 10:12:34.283165 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.283174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.283215 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:12:34.283578 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.283590 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.283635 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.283793 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.284630 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.285671 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.286731 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.287380 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.287498 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.290231 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.290235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.290235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.290248 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.292118 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.292482 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.292485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.292486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.292608 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.292830 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.292831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.292831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.293078 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.293258 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.293326 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.293480 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260811 10:12:34.308378 19843 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:34.308914 19843 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:34.312831 19843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Generated new TSK 0
I20260811 10:12:34.313014 19843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
I20260811 10:12:34.344781 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:34.428622 19870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:34.428828 19870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:34.428853 19870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:34.430087 19870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:34.430132 19870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:34.430158 19870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:34.431600 19870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:34.431870 19870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:34.432125 19870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:34.437187 19884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.437248 19888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.437198 19886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.437495 19870 server_base.cc:1034] running on GCE node
I20260811 10:12:34.437646 19870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:34.437903 19870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:34.439054 19870 hybrid_clock.cc:648] HybridClock initialized: now 1786443154439036 us; error 38 us; skew 500 ppm
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:34.441593 19870 init.cc:377] Logged in from keytab as oryx/127.8.238.1@KRBTEST.COM (short username oryx)
I20260811 10:12:34.442054 19870 webserver.cc:460] Webserver started at http://127.8.238.1:45991/ using document root <none> and password file <none>
I20260811 10:12:34.442279 19870 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:34.442349 19870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:34.442502 19870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:34.443559 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "2092d73a6ea24855ab50b2d1d3c2d057"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "39eb396cf0489a252f5f77672859f36c"
server_key_iv: "15a6ef3443dce8f13ceaf35d4ca6d436"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.443962 19870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "2092d73a6ea24855ab50b2d1d3c2d057"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "39eb396cf0489a252f5f77672859f36c"
server_key_iv: "15a6ef3443dce8f13ceaf35d4ca6d436"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.445303 19870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:34.446053 19905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.446221 19870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:34.446332 19870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "2092d73a6ea24855ab50b2d1d3c2d057"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "39eb396cf0489a252f5f77672859f36c"
server_key_iv: "15a6ef3443dce8f13ceaf35d4ca6d436"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.446408 19870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:34.462702 19870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:34.464648 19870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:34.464820 19870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:34.465075 19870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:34.465440 19870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:34.465498 19870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.465539 19870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:34.465569 19870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.471951 19870 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:36539
I20260811 10:12:34.472057 20093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:36539 every 8 connection(s)
I20260811 10:12:34.472358 19870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:34.480038 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 19870
I20260811 10:12:34.480157 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:34.480386 9144 external_mini_cluster.cc:1428] Setting key 13c11346da62b00f05755d4d0273d946
I20260811 10:12:34.481686 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.473086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:41911 (local address 127.8.238.62:43017)
0811 10:12:34.473135 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.473137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.473370 (+ 233us) server_negotiation.cc:407] Connection header received
0811 10:12:34.473856 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.473857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.473893 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.473944 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.474851 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.475390 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.476372 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.476586 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.476643 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.478752 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.478753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.478754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.478761 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.480166 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.480671 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.480673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.480674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.480738 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.480984 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.480985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.480985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.481109 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.481164 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.481451 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.481552 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:34.481707 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.473050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:41911)
0811 10:12:34.473229 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:34.473273 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:34.473774 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:34.474005 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:34.474009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:34.474582 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:34.474773 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.474778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.475477 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.475478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.476286 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.476288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.476666 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.476667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.476768 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.477210 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:34.477218 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:34.478642 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:34.480281 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.480289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.480291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.480590 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.480809 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.480810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.480811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.480917 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.481175 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:34.481182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:34.481317 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 10:12:34.481403 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:12:34.481532 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260811 10:12:34.482455 20095 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:34.482545 20095 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:34.482744 20095 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:34.483371 19762 ts_manager.cc:194] Registered new tserver with Master: 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539)
I20260811 10:12:34.484122 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:41911
WARNING: no policy specified for oryx/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
I20260811 10:12:34.510202 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:34.593881 20101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:34.594080 20101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:34.594113 20101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:34.595299 20101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:34.595341 20101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:34.595367 20101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:34.596832 20101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:34.597090 20101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:34.597322 20101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:34.602458 20107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.602597 20101 server_base.cc:1034] running on GCE node
W20260811 10:12:34.602450 20109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.602487 20111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.602893 20101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:34.603183 20101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:34.604337 20101 hybrid_clock.cc:648] HybridClock initialized: now 1786443154604318 us; error 50 us; skew 500 ppm
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:34.606875 20101 init.cc:377] Logged in from keytab as oryx/127.8.238.2@KRBTEST.COM (short username oryx)
I20260811 10:12:34.607331 20101 webserver.cc:460] Webserver started at http://127.8.238.2:36799/ using document root <none> and password file <none>
I20260811 10:12:34.607544 20101 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:34.607585 20101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:34.607676 20101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:34.608734 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "cf56422974ea423085ae144260bca2e8"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "2e4f5290130a7be595c573416bd732d6"
server_key_iv: "72765a881bb7ac0f56297640f7ec83ac"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.609047 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "cf56422974ea423085ae144260bca2e8"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "2e4f5290130a7be595c573416bd732d6"
server_key_iv: "72765a881bb7ac0f56297640f7ec83ac"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.610381 20101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:34.611075 20123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.611268 20101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:34.611392 20101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "cf56422974ea423085ae144260bca2e8"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "2e4f5290130a7be595c573416bd732d6"
server_key_iv: "72765a881bb7ac0f56297640f7ec83ac"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.611474 20101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:34.640087 20101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:34.642158 20101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:34.642334 20101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:34.642603 20101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:34.642956 20101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:34.643018 20101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.643061 20101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:34.643105 20101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.649724 20101 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:44321
I20260811 10:12:34.649770 20318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:44321 every 8 connection(s)
I20260811 10:12:34.650118 20101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:34.655719 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 20101
I20260811 10:12:34.655856 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:34.656086 9144 external_mini_cluster.cc:1428] Setting key 046578ba392051cfbfef596b41fd18fc
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:34.660329 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.650898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:41821 (local address 127.8.238.62:43017)
0811 10:12:34.650974 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.650977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.651186 (+ 209us) server_negotiation.cc:407] Connection header received
0811 10:12:34.651679 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.651680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.651691 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.651760 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.652725 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.653236 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.654182 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.654375 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.654428 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.657006 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.657008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.657008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.657020 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.658556 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.659094 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.659096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.659096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.659174 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.659441 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.659442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.659443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.659608 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.659699 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.660096 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.660205 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:12:34.660356 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.650884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:41821)
0811 10:12:34.651068 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:34.651104 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:34.651593 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:34.651859 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:34.651863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:34.652436 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:34.652642 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.652647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.653316 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.653317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.654107 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.654109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.654447 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.654448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.654557 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.655060 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:34.655068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:34.656786 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:34.658647 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.658657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.658659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.659007 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.659252 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.659253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.659253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.659370 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.659718 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:34.659725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:34.659945 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 10:12:34.660045 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 10:12:34.660185 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260811 10:12:34.660936 20319 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:34.661053 20319 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:34.661274 20319 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:34.661749 19762 ts_manager.cc:194] Registered new tserver with Master: cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
I20260811 10:12:34.662185 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:41821
WARNING: no policy specified for oryx/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
I20260811 10:12:34.684811 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:34.768615 20330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:34.768813 20330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:34.768836 20330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:34.769989 20330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:34.770030 20330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:34.770056 20330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:34.771524 20330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:34.771793 20330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:34.772058 20330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:34.777100 20344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.777100 20340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.777088 20341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.777328 20330 server_base.cc:1034] running on GCE node
I20260811 10:12:34.777617 20330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:34.777884 20330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:34.779052 20330 hybrid_clock.cc:648] HybridClock initialized: now 1786443154779032 us; error 30 us; skew 500 ppm
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:34.781693 20330 init.cc:377] Logged in from keytab as oryx/127.8.238.3@KRBTEST.COM (short username oryx)
I20260811 10:12:34.782123 20330 webserver.cc:460] Webserver started at http://127.8.238.3:42807/ using document root <none> and password file <none>
I20260811 10:12:34.782370 20330 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:34.782441 20330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:34.782601 20330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:34.783659 20330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "1e4781dbf11449c492a52e214a205408"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "60e03913dfbccd67f5a646a19c889392"
server_key_iv: "6a26a39ab2c80c933ad775c0e5ac3cdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.784080 20330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "1e4781dbf11449c492a52e214a205408"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "60e03913dfbccd67f5a646a19c889392"
server_key_iv: "6a26a39ab2c80c933ad775c0e5ac3cdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.785470 20330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:34.786231 20357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.786443 20330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:34.786545 20330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "1e4781dbf11449c492a52e214a205408"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "60e03913dfbccd67f5a646a19c889392"
server_key_iv: "6a26a39ab2c80c933ad775c0e5ac3cdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.786628 20330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:34.794700 20330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:34.796590 20330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:34.796746 20330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:34.796964 20330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:34.797269 20330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:34.797353 20330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.797394 20330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:34.797437 20330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.803375 20330 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33815
I20260811 10:12:34.803414 20539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33815 every 8 connection(s)
I20260811 10:12:34.803859 20330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:34.810266 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 20330
I20260811 10:12:34.810360 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:34.810923 9144 external_mini_cluster.cc:1428] Setting key 4aca1339f596e74ddf8c6c8bb6a2b9b8
I20260811 10:12:34.813400 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.804569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:40053 (local address 127.8.238.62:43017)
0811 10:12:34.804659 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.804662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.804851 (+ 189us) server_negotiation.cc:407] Connection header received
0811 10:12:34.805310 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.805311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.805321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.805372 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.806261 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.806780 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.807746 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.807934 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.807994 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.810123 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.810126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.810126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.810135 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.811968 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.812378 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.812379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.812379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.812440 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.812666 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.812667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.812668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.812803 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.812915 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.813202 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.813292 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:12:34.813614 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.804539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:40053)
0811 10:12:34.804730 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:34.804764 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:34.805220 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:34.805422 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:34.805426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:34.805985 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:34.806168 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.806172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.806868 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.806869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.807658 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.807660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.808012 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.808013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.808119 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.808563 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:34.808571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:34.809974 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:34.812040 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.812047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.812049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.812310 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.812484 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.812484 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.812484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.812589 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.812908 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:34.812915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:34.813062 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 10:12:34.813176 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 10:12:34.813371 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260811 10:12:34.814071 20542 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:34.814178 20542 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:34.814368 20542 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:34.814713 19762 ts_manager.cc:194] Registered new tserver with Master: 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:34.815107 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:40053
I20260811 10:12:34.822971 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.238.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.61.keytab.
Entry for principal oryx/127.8.238.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.61.keytab.
I20260811 10:12:34.850984 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:37013
--webserver_interface=127.8.238.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.61.keytab
--principal=oryx/127.8.238.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:43017,127.8.238.61:37013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:34.937079 20549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:34.937297 20549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:34.937321 20549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:34.938583 20549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:34.938628 20549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:34.938643 20549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:34.938656 20549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:34.938668 20549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:34.940148 20549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:43017,127.8.238.61:37013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.61.keytab
--principal=oryx/127.8.238.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:37013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.238.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
W20260811 10:12:34.940243 20549 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.238.62:43017,127.8.238.61:37013'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 10:12:34.940424 20549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:34.940671 20549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:34.945523 20557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.945529 20558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:34.945529 20564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:34.945827 20549 server_base.cc:1034] running on GCE node
I20260811 10:12:34.946151 20549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:34.946487 20549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:34.947672 20549 hybrid_clock.cc:648] HybridClock initialized: now 1786443154947654 us; error 28 us; skew 500 ppm
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:34.950424 20549 init.cc:377] Logged in from keytab as oryx/127.8.238.61@KRBTEST.COM (short username oryx)
I20260811 10:12:34.950907 20549 webserver.cc:460] Webserver started at http://127.8.238.61:41937/ using document root <none> and password file <none>
I20260811 10:12:34.951157 20549 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:34.951229 20549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:34.951370 20549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:34.952481 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/instance:
uuid: "be7a19682dfc40eb99d4093b863a61e3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "e0fd93c07d0772fd24691c705e524354"
server_key_iv: "c6613cfadf92db3fff12014a1615db51"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.952855 20549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal/instance:
uuid: "be7a19682dfc40eb99d4093b863a61e3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "e0fd93c07d0772fd24691c705e524354"
server_key_iv: "c6613cfadf92db3fff12014a1615db51"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.954207 20549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:34.955006 20572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:34.955199 20549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:34.955297 20549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "be7a19682dfc40eb99d4093b863a61e3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "e0fd93c07d0772fd24691c705e524354"
server_key_iv: "c6613cfadf92db3fff12014a1615db51"
server_key_version: "encryptionkey@0"
I20260811 10:12:34.955353 20549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:34.971011 20549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:34.973169 20549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:34.973284 20549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:34.977561 20549 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.61:37013
I20260811 10:12:34.977595 20655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.61:37013 every 8 connection(s)
I20260811 10:12:34.977947 20549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
I20260811 10:12:34.978955 20656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:34.980583 20656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.61@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:34.986711 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 20549
I20260811 10:12:34.990506 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.981332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.8.238.62:43017)
0811 10:12:34.981408 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.981410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.981643 (+ 233us) server_negotiation.cc:407] Connection header received
0811 10:12:34.982199 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.982201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.982216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.982263 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.982939 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.983458 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.984549 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.984749 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.984836 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.987365 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.987367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.987367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.987376 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.989012 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.989436 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.989437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.989438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.989509 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.989774 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.989775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.989776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.990001 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.990075 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.990342 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.990430 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:12:34.990736 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.981327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45538)
0811 10:12:34.981520 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:34.981558 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:34.982075 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:34.982367 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:34.982372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:34.982641 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:34.982835 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.982840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.983571 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.983573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.984430 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.984433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.984849 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.984850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.984977 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.985463 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:34.985478 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:34.987261 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:34.989092 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.989100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.989102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.989326 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.989581 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.989582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.989582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.989703 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.990071 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:34.990077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:34.990202 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 10:12:34.990294 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:12:34.990489 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260811 10:12:34.991520 20656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
Aug 11 10:12:34 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.61@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
I20260811 10:12:34.997155 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.987332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.8.238.61:37013)
0811 10:12:34.987488 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.987493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.987541 (+ 48us) server_negotiation.cc:407] Connection header received
0811 10:12:34.987606 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.987608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.987644 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.987756 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.988591 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.989357 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.990386 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.990940 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.991020 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.994012 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.994019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.994019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.994026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.995691 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.996156 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.996159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.996160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.996230 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.996461 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.996462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.996462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.996656 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.996834 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.996923 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.997023 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260811 10:12:34.999681 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.991803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56680)
0811 10:12:34.991844 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:34.991855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:34.991921 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:34.992257 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:34.992258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:34.992470 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:34.992529 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.992531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.993589 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.993591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.994275 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:34.994279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.994887 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.994889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:34.994970 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.995297 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:34.995302 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:34.996476 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:34.998784 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.998785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.998785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.998997 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.999229 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:34.999229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:34.999230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:34.999283 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:34.999480 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:34.999481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:34.999511 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 10:12:34.999553 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 10:12:34.999624 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:12:34.999891 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:34.991908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.8.238.61:37013)
0811 10:12:34.992084 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:34.992086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:34.992096 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:34.992119 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:34.992120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:34.992132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:34.992219 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:34.992620 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.993466 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.994358 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:34.994776 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:34.994892 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:34.996565 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:34.996566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:34.996566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:34.996573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:34.998712 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.999086 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.999087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.999087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.999158 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:34.999329 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:34.999329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:34.999330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:34.999432 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:34.999485 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:34.999578 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 10:12:34.999674 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260811 10:12:35.001277 20656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Bootstrap starting.
I20260811 10:12:35.001965 20656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:35.002278 20656 log.cc:824] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:35.003057 20656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: No bootstrap required, opened a new log
I20260811 10:12:35.004658 20656 raft_consensus.cc:348] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } }
I20260811 10:12:35.004787 20656 raft_consensus.cc:374] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:35.004846 20656 raft_consensus.cc:729] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7a19682dfc40eb99d4093b863a61e3, State: Initialized, Role: FOLLOWER
I20260811 10:12:35.004983 20656 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } }
I20260811 10:12:35.005292 20667 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } }
I20260811 10:12:35.005405 20667 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:35.005349 20656 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 10:12:35.007566 20684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:12:35.007625 20684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:12:35.009359 20549 master_runner.cc:428] Detected that this master be7a19682dfc40eb99d4093b863a61e3 is joining an existing cluster
I20260811 10:12:35.009433 20549 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.238.61:37013
I20260811 10:12:35.016990 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.010423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45540)
0811 10:12:35.010579 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.010589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.010652 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.010821 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.010822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.011056 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.011129 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.011131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.012092 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.012093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.012714 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.012716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.013091 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.013092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.013156 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.013700 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.013705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.014113 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.015821 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.015822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.015823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.016061 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.016276 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.016277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.016277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.016342 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.016606 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.016607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.016652 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.016770 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.016879 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260811 10:12:35.017004 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.010457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45540 (local address 127.8.238.62:43017)
0811 10:12:35.010504 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.010506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.010657 (+ 151us) server_negotiation.cc:407] Connection header received
0811 10:12:35.010717 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.010717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.010728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.010779 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.011207 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.012004 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.012803 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.013019 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.013093 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.014184 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.014186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.014186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.014193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.015695 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.016135 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.016137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.016137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.016199 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.016410 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.016411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.016411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.016530 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.016604 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.016814 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.016899 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:35.019310 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.010494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56688)
0811 10:12:35.010799 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.010809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.010875 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.011039 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.011040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.011256 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.011333 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.011335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.012228 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.012230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.013168 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.013171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.013745 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.013746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.013807 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.014456 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.014460 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.014838 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.017450 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.017451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.017451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.017672 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.018554 (+ 882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.018555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.018556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.018648 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.018962 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.018962 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.018998 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.019089 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.019195 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":151,"threads_started":1}
I20260811 10:12:35.019589 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.010725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.8.238.61:37013)
0811 10:12:35.010760 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.010761 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.010877 (+ 116us) server_negotiation.cc:407] Connection header received
0811 10:12:35.010938 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.010939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.010949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.011013 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.011422 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.012147 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.013264 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.013612 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.013699 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.014927 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.014929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.014929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.014936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.017313 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.018103 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.018104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.018104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.018167 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.018738 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.018739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.018739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.018868 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.018915 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.019409 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.019506 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.62@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
I20260811 10:12:35.026672 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.019628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.8.238.61:37013)
0811 10:12:35.019677 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.019679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.020045 (+ 366us) server_negotiation.cc:407] Connection header received
0811 10:12:35.020150 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.020151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.020161 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.020206 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.020628 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.021077 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.021839 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.022037 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.022109 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.024143 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.024144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.024144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.024151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.025453 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.025845 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.025846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.025846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.025908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.026124 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.026125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.026125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.026230 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.026296 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.026514 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.026581 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:12:35.026700 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.019571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56702)
0811 10:12:35.019755 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.019984 (+ 229us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.020079 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.020222 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.020226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.020500 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.020546 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.020547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.021159 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.021160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.021752 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.021754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.022111 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.022112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.022197 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.022692 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.022701 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.024043 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.025554 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.025561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.025564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.025767 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.025989 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.025990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.025990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.026053 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.026304 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.026312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.026362 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.026452 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.026580 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260811 10:12:35.027269 19762 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.238.61:37013: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3"
member_type: NON_VOTER
last_known_addr {
host: "127.8.238.61"
port: 37013
}
attrs {
promote: true
}
}
dest_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3"
cas_config_opid_index: -1
I20260811 10:12:35.027622 19762 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } }
I20260811 10:12:35.028151 20700 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.028532 20616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:35.029302 20700 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.238.61:37013
I20260811 10:12:35.029515 20616 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 10:12:35.029500 20698 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.029598 20698 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.029791 20549 master.cc:537] Master@127.8.238.61:37013 shutting down...
I20260811 10:12:35.029865 20698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 10:12:35.030725 19744 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 10:12:35.031210 19744 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 -> Peer be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61:37013): Couldn't send request to peer be7a19682dfc40eb99d4093b863a61e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 10:12:35.033391 20549 raft_consensus.cc:2229] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:35.033501 20549 raft_consensus.cc:2258] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:35.033558 20549 tablet_replica.cc:330] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: stopping tablet replica
I20260811 10:12:35.046924 20549 master.cc:552] Master@127.8.238.61:37013 shutdown complete.
I20260811 10:12:35.046994 20549 master_runner.cc:305] Clearing existing system tablet
I20260811 10:12:35.047122 20549 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.047184 20549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.047885 20549 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260811 10:12:35.048453 20705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.048580 20549 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.048637 20549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.048663 20549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "be7a19682dfc40eb99d4093b863a61e3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "e0fd93c07d0772fd24691c705e524354"
server_key_iv: "c6613cfadf92db3fff12014a1615db51"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.049463 20549 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:35.050879 20549 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 10:12:35.050964 20549 log.cc:1196] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 10:12:35.051188 20549 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Deleting consensus metadata
I20260811 10:12:35.051316 20549 master_runner.cc:315] Copying system tablet from 127.8.238.62:43017
I20260811 10:12:35.051929 20549 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: tablet copy: Beginning tablet copy session from remote peer at address 127.8.238.62:43017
I20260811 10:12:35.061358 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.052400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45550 (local address 127.8.238.62:43017)
0811 10:12:35.052455 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.052457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.055353 (+ 2896us) server_negotiation.cc:407] Connection header received
0811 10:12:35.055441 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.055442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.055463 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.055531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.055932 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.056461 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.057214 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.057412 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.057492 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.058593 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.058595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.058595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.058602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.060126 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.060516 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.060517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.060518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.060578 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.060823 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.060825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.060825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.060934 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.061014 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.061179 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.061260 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:35.061380 20716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.052297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45550)
0811 10:12:35.055256 (+ 2959us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.055266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.055368 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.055537 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.055538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.055769 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.055846 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.055848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.056550 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.056552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.057121 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.057123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.057493 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.057494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.057569 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.058122 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.058145 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.058500 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.060228 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.060229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.060230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.060429 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.060653 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.060653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.060654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.060721 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.061005 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.061007 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.061054 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.061152 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.061261 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2926,"thread_start_us":83,"threads_started":1}
I20260811 10:12:35.061991 19793 tablet_copy_service.cc:140] P b6c28fcf12f4425d919b8ff54d0fd0b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer be7a19682dfc40eb99d4093b863a61e3 ({username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:45550)
I20260811 10:12:35.062112 19793 tablet_copy_service.cc:161] P b6c28fcf12f4425d919b8ff54d0fd0b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer be7a19682dfc40eb99d4093b863a61e3 at {username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:45550: session id = be7a19682dfc40eb99d4093b863a61e3-00000000000000000000000000000000
I20260811 10:12:35.062968 19793 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 10:12:35.063673 20549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 10:12:35.064771 20549 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: tablet copy: Starting download of 0 data blocks...
I20260811 10:12:35.064925 20549 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: tablet copy: Starting download of 1 WAL segments...
I20260811 10:12:35.065956 20549 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 10:12:35.066803 19793 tablet_copy_service.cc:342] P b6c28fcf12f4425d919b8ff54d0fd0b3: Request end of tablet copy session be7a19682dfc40eb99d4093b863a61e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:45550
I20260811 10:12:35.066866 19793 tablet_copy_service.cc:433] P b6c28fcf12f4425d919b8ff54d0fd0b3: ending tablet copy session be7a19682dfc40eb99d4093b863a61e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer be7a19682dfc40eb99d4093b863a61e3
I20260811 10:12:35.069545 20549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.070762 20720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.070829 20723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.070770 20718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.071013 20549 server_base.cc:1034] running on GCE node
I20260811 10:12:35.071179 20549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.071350 20549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.072499 20549 hybrid_clock.cc:648] HybridClock initialized: now 1786443155072487 us; error 23 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.073905 20549 init.cc:377] Logged in from keytab as oryx/127.8.238.61@KRBTEST.COM (short username oryx)
I20260811 10:12:35.074198 20549 webserver.cc:460] Webserver started at http://127.8.238.61:39705/ using document root <none> and password file <none>
I20260811 10:12:35.074317 20549 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.074344 20549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.075033 20549 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:35.075574 20733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.075704 20549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.075798 20549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "be7a19682dfc40eb99d4093b863a61e3"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "e0fd93c07d0772fd24691c705e524354"
server_key_iv: "c6613cfadf92db3fff12014a1615db51"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.075860 20549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.088054 20549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.089814 20549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.093851 20549 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.61:37013
I20260811 10:12:35.094106 20819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.61:37013 every 8 connection(s)
I20260811 10:12:35.094272 20549 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
I20260811 10:12:35.095377 20820 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:12:35.096022 20820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Bootstrap starting.
I20260811 10:12:35.099457 20820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:12:35.099779 20820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Bootstrap complete.
I20260811 10:12:35.100032 20820 raft_consensus.cc:348] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } }
I20260811 10:12:35.100142 20820 raft_consensus.cc:729] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: be7a19682dfc40eb99d4093b863a61e3, State: Initialized, Role: LEARNER
I20260811 10:12:35.100188 20820 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } }
I20260811 10:12:35.100404 20820 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:35.100415 20825 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.100649 20825 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 10:12:35.103080 20846 catalog_manager.cc:1259] Loaded cluster ID: 925ee9ecc2c34d71bcb6c262c9cc4df7
I20260811 10:12:35.103154 20846 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: loading cluster ID for follower catalog manager: success
I20260811 10:12:35.104298 20846 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: acquiring CA information for follower catalog manager: success
I20260811 10:12:35.104594 20846 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:12:35.107133 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.098775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.8.238.61:37013)
0811 10:12:35.098914 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.098916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.098990 (+ 74us) server_negotiation.cc:407] Connection header received
0811 10:12:35.099060 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.099061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.099073 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.099165 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.099829 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.100838 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.101969 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.102390 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.102478 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.104098 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.104100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.104100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.104108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.105917 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.106279 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.106280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.106281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.106355 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.106556 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.106557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.106558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.106701 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.106839 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.106909 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.107018 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for oryx/127.8.238.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.60.keytab.
Entry for principal oryx/127.8.238.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.60.keytab.
I20260811 10:12:35.135107 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:42069
--webserver_interface=127.8.238.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.60.keytab
--principal=oryx/127.8.238.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:35.221791 20853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:35.221997 20853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:35.222019 20853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:35.223085 20853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:35.223117 20853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:35.223131 20853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:35.223145 20853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:35.223156 20853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:35.224586 20853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.60.keytab
--principal=oryx/127.8.238.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:42069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.238.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:35.224777 20853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:35.225018 20853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.229715 20858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.229714 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.229808 20861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.230189 20853 server_base.cc:1034] running on GCE node
I20260811 10:12:35.230495 20853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.230726 20853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.231868 20853 hybrid_clock.cc:648] HybridClock initialized: now 1786443155231859 us; error 28 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.234427 20853 init.cc:377] Logged in from keytab as oryx/127.8.238.60@KRBTEST.COM (short username oryx)
I20260811 10:12:35.234843 20853 webserver.cc:460] Webserver started at http://127.8.238.60:38791/ using document root <none> and password file <none>
I20260811 10:12:35.235087 20853 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.235157 20853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.235316 20853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:35.236438 20853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/instance:
uuid: "32d69584b4f14f15ba0b037aa1d3078d"
format_stamp: "Formatted at 2026-08-11 10:12:35 on dist-test-slave-7f01"
server_key: "4244151908381e23ab10d33609ae4317"
server_key_iv: "c5fccca58cf085ff9aaeb540ca8513d6"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.236795 20853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal/instance:
uuid: "32d69584b4f14f15ba0b037aa1d3078d"
format_stamp: "Formatted at 2026-08-11 10:12:35 on dist-test-slave-7f01"
server_key: "4244151908381e23ab10d33609ae4317"
server_key_iv: "c5fccca58cf085ff9aaeb540ca8513d6"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.238169 20853 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:12:35.238914 20875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.239096 20853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:35.239205 20853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "32d69584b4f14f15ba0b037aa1d3078d"
format_stamp: "Formatted at 2026-08-11 10:12:35 on dist-test-slave-7f01"
server_key: "4244151908381e23ab10d33609ae4317"
server_key_iv: "c5fccca58cf085ff9aaeb540ca8513d6"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.239279 20853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.252686 20853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.254552 20853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:35.254720 20853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.259529 20853 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.60:42069
I20260811 10:12:35.259604 20964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.60:42069 every 8 connection(s)
I20260811 10:12:35.260027 20853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
I20260811 10:12:35.260836 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 20853
I20260811 10:12:35.261165 20967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:35.263831 20967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.60@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:35.272753 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.261560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.8.238.60:42069)
0811 10:12:35.261745 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.261755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.261788 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:12:35.262579 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.262590 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.262658 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.262812 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.263457 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.264879 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.265894 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.266478 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.266554 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.269371 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.269378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.269378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.269384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.270996 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.271381 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.271385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.271385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.271484 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.271700 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.271701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.271702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.272048 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.272231 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.272379 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.272515 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260811 10:12:35.273088 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.264500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45562)
0811 10:12:35.264698 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.264736 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.264798 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.265021 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.265024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.265384 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.265543 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.265546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.266450 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.266451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.267603 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.267607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.268039 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.268040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.268123 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.268599 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.268608 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.270208 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.271849 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.271854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.271856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.272111 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.272322 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.272322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.272323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.272391 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.272656 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.272661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.272710 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.272869 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.272973 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":133,"threads_started":1}
I20260811 10:12:35.273111 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.264792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.8.238.62:43017)
0811 10:12:35.264871 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.264873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.264884 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:35.264913 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.264914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.264926 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.264987 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.265626 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.266273 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.267707 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.267954 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.268007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.270318 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.270320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.270320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.270328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.271785 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.272179 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.272180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.272181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.272250 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.272469 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.272470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.272470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.272605 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.272660 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.272909 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.273021 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 10:12:35.273697 20967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.60@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
I20260811 10:12:35.280537 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.273958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56734)
0811 10:12:35.274010 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.274027 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.274058 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.274265 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.274265 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.274393 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.274436 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.274437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.275094 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.275096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.275690 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.275693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.276122 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.276123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.276191 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.276542 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.276547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.277740 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.279361 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.279362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.279362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.279603 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.279868 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.279869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.279869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.279935 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.280214 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.280215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.280256 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.280342 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.280436 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:12:35.280541 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.274065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.8.238.61:37013)
0811 10:12:35.274136 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.274138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.274146 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:35.274173 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.274176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.274190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.274253 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.274514 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.275008 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.275838 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.276044 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.276100 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.277833 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.277835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.277835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.277849 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.279256 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.279684 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.279685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.279685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.279792 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.280014 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.280015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.280016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.280135 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.280207 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.280371 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.280455 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:12:35.281138 20967 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.60@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
I20260811 10:12:35.287184 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.281373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.0.0.1:39350)
0811 10:12:35.281529 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.281537 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.281575 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.281667 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.281668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.281773 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.281800 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.281801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.282416 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.282417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.282856 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.282857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.283176 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.283177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.283211 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.283514 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.283518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.284708 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.286233 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.286234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.286234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.286429 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.286620 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.286620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.286621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.286700 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.286945 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.286946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.286989 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.287053 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.287126 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 10:12:35.287322 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.281497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.8.238.60:42069)
0811 10:12:35.281544 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.281546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.281565 (+ 19us) server_negotiation.cc:407] Connection header received
0811 10:12:35.281612 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.281613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.281620 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.281665 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.281897 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.282341 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.282926 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.283111 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.283164 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.284784 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.284786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.284786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.284792 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.286093 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.286503 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.286504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.286505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.286556 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.286778 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.286779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.286779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.286878 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.286931 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.287068 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.287171 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":29}
I20260811 10:12:35.288916 20967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Bootstrap starting.
I20260811 10:12:35.290114 20967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:35.290729 20967 log.cc:824] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:35.291569 20967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: No bootstrap required, opened a new log
I20260811 10:12:35.293757 20967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } }
I20260811 10:12:35.293958 20967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:35.294016 20967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d69584b4f14f15ba0b037aa1d3078d, State: Initialized, Role: FOLLOWER
I20260811 10:12:35.294174 20967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } }
I20260811 10:12:35.294567 20978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } } }
I20260811 10:12:35.294643 20967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:35.294654 20978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:12:35.296927 20997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:12:35.296995 20997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:12:35.299556 20853 master_runner.cc:428] Detected that this master 32d69584b4f14f15ba0b037aa1d3078d is joining an existing cluster
I20260811 10:12:35.299619 20853 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.238.60:42069
I20260811 10:12:35.324496 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 19870
I20260811 10:12:35.329414 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.8.238.61:37013)
0811 10:12:35.301425 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.301427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.302090 (+ 663us) server_negotiation.cc:407] Connection header received
0811 10:12:35.302144 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.302145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.302155 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.302206 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.302996 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303821 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.305527 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.305835 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.305908 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.326074 (+ 20166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.326076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.326076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.326086 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.328023 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.328473 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.328474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.328475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.328538 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.328750 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.328751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.328751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.328908 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.328956 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.329198 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.329290 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:35.329885 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56750)
0811 10:12:35.302035 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.302044 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.302094 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.302292 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.302294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.302649 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.302879 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.302882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303922 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.303923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.305427 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.305430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.305926 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.305927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.305996 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.306493 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.306498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.325111 (+ 18613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.328129 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.328131 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.328132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.328378 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.328615 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.328616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.328617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.328677 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.329009 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.329010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.329076 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.329526 (+ 450us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.329674 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":98,"threads_started":1}
I20260811 10:12:35.333067 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.8.238.60:42069)
0811 10:12:35.301187 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.301189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.301867 (+ 678us) server_negotiation.cc:407] Connection header received
0811 10:12:35.301918 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.301919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.301927 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.301951 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.302686 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303417 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.304587 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.304855 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.304892 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.306154 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.306155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.306155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.306162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.325932 (+ 19770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.332163 (+ 6231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.332164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.332164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.332246 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.332490 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.332491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.332491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.332619 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.332662 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.332881 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.332968 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:12:35.333424 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.0.0.1:39354)
0811 10:12:35.301777 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.301789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.301859 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.302176 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.302176 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.302447 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.302569 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.302572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303500 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.303500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.304497 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.304500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.304972 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.304972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.305033 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.305565 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.305569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.306070 (+ 501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.331313 (+ 25243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.331314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.331314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.331623 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.332352 (+ 729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.332353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.332353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.332418 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.332704 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.332705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.332746 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.333197 (+ 451us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.333306 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":24,"thread_start_us":229,"threads_started":1}
I20260811 10:12:35.351234 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45568)
0811 10:12:35.301777 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.301789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.301859 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.302036 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.302037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.302461 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.302657 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.302660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303575 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.303576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.304616 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.304617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.305111 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.305112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.305240 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.324391 (+ 19151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.324397 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.332049 (+ 7652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.350047 (+ 17998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.350049 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.350049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.350256 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.350501 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.350502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.350503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.350570 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.350852 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.350854 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.350906 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.350991 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.351114 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":4,"thread_start_us":352,"threads_started":1}
I20260811 10:12:35.351218 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.301077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.8.238.62:43017)
0811 10:12:35.301123 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.301125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.301903 (+ 778us) server_negotiation.cc:407] Connection header received
0811 10:12:35.301931 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.301932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.301945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.302018 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.302745 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.303487 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.304696 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.304994 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.305032 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.346660 (+ 41628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.346663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.346664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.346675 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.349924 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.350350 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.350352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.350353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.350422 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.350634 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.350635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.350636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.350767 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.350850 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.351019 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.351113 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:35.355221 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:36539
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:43017
--webserver_port=45991
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443154, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.62@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
I20260811 10:12:35.361231 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.353862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.0.0.1:39368)
0811 10:12:35.353918 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.353929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.353972 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.354116 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.354117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.354278 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.354326 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.354329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.355135 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.355137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.356009 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.356013 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.356552 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.356553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.356630 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.357134 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.357140 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.358394 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.360114 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.360115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.360116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.360301 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.360520 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.360521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.360521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.360590 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.360886 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.360887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.360936 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.361023 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.361132 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:12:35.361230 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.353914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.8.238.60:42069)
0811 10:12:35.353954 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.353957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.353965 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:35.354044 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.354045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.354055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.354111 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.354415 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.355030 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.356109 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.356428 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.356492 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.358492 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.358494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.358494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.358502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.360009 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.360385 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.360386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.360386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.360445 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.360659 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.360660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.360660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.360812 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.360893 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.361050 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.361131 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:35.362304 19762 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.238.60:42069: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d"
member_type: NON_VOTER
last_known_addr {
host: "127.8.238.60"
port: 42069
}
attrs {
promote: true
}
}
dest_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3"
cas_config_opid_index: 5
I20260811 10:12:35.362619 19762 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
I20260811 10:12:35.363027 20698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.363967 20920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:35.365288 20920 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 10:12:35.365610 20700 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:35.366107 20698 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.238.60:42069
I20260811 10:12:35.366176 20698 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.366236 20698 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.366564 20853 master.cc:537] Master@127.8.238.60:42069 shutting down...
I20260811 10:12:35.367877 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.363201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.8.238.61:37013)
0811 10:12:35.363249 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.363251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.363503 (+ 252us) server_negotiation.cc:407] Connection header received
0811 10:12:35.363604 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.363605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.363632 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.363673 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.363910 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.364700 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.366625 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.367303 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.367357 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.367445 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.367693 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.367763 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:35.368197 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.363078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:56758)
0811 10:12:35.363450 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.363462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.363531 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.363766 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.363767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.363768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.363820 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.363821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.364838 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.364839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.366520 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.366524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.367510 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.367511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.367586 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.367609 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.368017 (+ 408us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.368105 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355}
I20260811 10:12:35.368475 20778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 LEARNER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 10:12:35.368889 20700 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 10:12:35.369489 20778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.370100 20920 raft_consensus.cc:858] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
W20260811 10:12:35.370154 20920 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
I20260811 10:12:35.370651 20920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.370929 20825 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6c28fcf12f4425d919b8ff54d0fd0b3. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.371006 20825 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 10:12:35.371037 20825 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.371071 20825 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: LEARNER
I20260811 10:12:35.371513 20700 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: attempting to promote NON_VOTER be7a19682dfc40eb99d4093b863a61e3 to VOTER
I20260811 10:12:35.371686 20978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6c28fcf12f4425d919b8ff54d0fd0b3. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.371799 20978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 10:12:35.371841 20978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.371908 20978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 10:12:35.371716 20700 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
I20260811 10:12:35.371935 20978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: NON_VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: true } } }
I20260811 10:12:35.371973 20978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 10:12:35.372594 20778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 LEARNER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 10:12:35.372931 20698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 10:12:35.373077 19743 consensus_peers.cc:487] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 -> Peer 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60:42069): Couldn't send request to peer 32d69584b4f14f15ba0b037aa1d3078d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 10:12:35.373585 20698 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.374058 21018 mvcc.cc:205] Tried to move back new op lower bound from 7317271160063741952 to 7317271159954702336. Current Snapshot: MvccSnapshot[applied={T|T < 7317271159968075776}]
I20260811 10:12:35.374735 20698 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.374832 20698 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.375237 20853 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 10:12:35.375270 20700 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.375334 20853 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 10:12:35.375355 20700 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.375372 20853 tablet_replica.cc:330] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: stopping tablet replica
I20260811 10:12:35.375620 20778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.376485 21021 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.376653 21021 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:35.381088 20853 master.cc:552] Master@127.8.238.60:42069 shutdown complete.
I20260811 10:12:35.381151 20853 master_runner.cc:305] Clearing existing system tablet
I20260811 10:12:35.381294 20853 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.381356 20853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.381992 20853 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:35.382483 21031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.382614 20853 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.382663 20853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.382692 20853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "32d69584b4f14f15ba0b037aa1d3078d"
format_stamp: "Formatted at 2026-08-11 10:12:35 on dist-test-slave-7f01"
server_key: "4244151908381e23ab10d33609ae4317"
server_key_iv: "c5fccca58cf085ff9aaeb540ca8513d6"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.383373 20853 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:35.384876 20853 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 10:12:35.384971 20853 log.cc:1196] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 10:12:35.385232 20853 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Deleting consensus metadata
I20260811 10:12:35.385414 20853 master_runner.cc:315] Copying system tablet from 127.8.238.62:43017
I20260811 10:12:35.386133 20853 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: tablet copy: Beginning tablet copy session from remote peer at address 127.8.238.62:43017
I20260811 10:12:35.394148 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.386495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.8.238.62:43017)
0811 10:12:35.386549 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.386551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.387983 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 10:12:35.388069 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.388070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.388087 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.388149 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.388502 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.389057 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.389980 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.390179 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.390269 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.391410 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.391411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.391411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.391418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.392930 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.393324 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.393325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.393325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.393389 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.393600 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.393601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.393601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.393718 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.393791 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.393967 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.394049 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:35.394166 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.386453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:45576)
0811 10:12:35.387907 (+ 1454us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.387918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.388001 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.388155 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.388156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.388363 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.388416 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.388418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.389147 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.389149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.389896 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.389899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.390270 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.390271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.390358 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.390956 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.390962 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.391287 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.393034 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.393035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.393036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.393228 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.393463 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.393464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.393464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.393533 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.393794 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.393796 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.393848 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.393935 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.394051 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1431,"thread_start_us":86,"threads_started":1}
I20260811 10:12:35.394786 19793 tablet_copy_service.cc:140] P b6c28fcf12f4425d919b8ff54d0fd0b3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 32d69584b4f14f15ba0b037aa1d3078d ({username='oryx', principal='oryx/127.8.238.60@KRBTEST.COM'} at 127.0.0.1:45576)
I20260811 10:12:35.394870 19793 tablet_copy_service.cc:161] P b6c28fcf12f4425d919b8ff54d0fd0b3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 32d69584b4f14f15ba0b037aa1d3078d at {username='oryx', principal='oryx/127.8.238.60@KRBTEST.COM'} at 127.0.0.1:45576: session id = 32d69584b4f14f15ba0b037aa1d3078d-00000000000000000000000000000000
I20260811 10:12:35.395555 19793 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 10:12:35.396219 20853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 10:12:35.397282 20853 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: tablet copy: Starting download of 0 data blocks...
I20260811 10:12:35.397444 20853 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: tablet copy: Starting download of 1 WAL segments...
I20260811 10:12:35.398167 20853 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 10:12:35.398948 19793 tablet_copy_service.cc:342] P b6c28fcf12f4425d919b8ff54d0fd0b3: Request end of tablet copy session 32d69584b4f14f15ba0b037aa1d3078d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.238.60@KRBTEST.COM'} at 127.0.0.1:45576
I20260811 10:12:35.399013 19793 tablet_copy_service.cc:433] P b6c28fcf12f4425d919b8ff54d0fd0b3: ending tablet copy session 32d69584b4f14f15ba0b037aa1d3078d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 32d69584b4f14f15ba0b037aa1d3078d
I20260811 10:12:35.402477 20853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.403708 21039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.404855 20853 server_base.cc:1034] running on GCE node
W20260811 10:12:35.404963 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.408195 21040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.408394 20853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.408564 20853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.409703 20853 hybrid_clock.cc:648] HybridClock initialized: now 1786443155409712 us; error 47 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.411274 20853 init.cc:377] Logged in from keytab as oryx/127.8.238.60@KRBTEST.COM (short username oryx)
I20260811 10:12:35.411608 20853 webserver.cc:460] Webserver started at http://127.8.238.60:40199/ using document root <none> and password file <none>
I20260811 10:12:35.411759 20853 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.411803 20853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.412473 20853 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:35.413089 21058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.413254 20853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:35.413306 20853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "32d69584b4f14f15ba0b037aa1d3078d"
format_stamp: "Formatted at 2026-08-11 10:12:35 on dist-test-slave-7f01"
server_key: "4244151908381e23ab10d33609ae4317"
server_key_iv: "c5fccca58cf085ff9aaeb540ca8513d6"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.413381 20853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.421859 20853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.423708 20853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.427453 20853 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.60:42069
I20260811 10:12:35.427901 20853 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
I20260811 10:12:35.428201 21140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.60:42069 every 8 connection(s)
I20260811 10:12:35.428757 21143 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:12:35.429219 21143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Bootstrap starting.
I20260811 10:12:35.431315 21143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:12:35.431577 21143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: Bootstrap complete.
I20260811 10:12:35.431926 21143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
I20260811 10:12:35.432032 21143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 32d69584b4f14f15ba0b037aa1d3078d, State: Initialized, Role: LEARNER
I20260811 10:12:35.432085 21143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } }
I20260811 10:12:35.432317 21143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:35.432291 21150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.432528 21150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: LEARNER
I20260811 10:12:35.434106 21165 catalog_manager.cc:1259] Loaded cluster ID: 925ee9ecc2c34d71bcb6c262c9cc4df7
I20260811 10:12:35.434158 21165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: loading cluster ID for follower catalog manager: success
I20260811 10:12:35.435335 21165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: acquiring CA information for follower catalog manager: success
I20260811 10:12:35.435643 21165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 10:12:35.494201 21013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:35.494544 21013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:35.494586 21013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:35.496497 21013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:35.496572 21013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:35.496618 21013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:35.499107 21013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:36539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=45991
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:35.499405 21013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:35.499683 21013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.507208 21171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.507226 21174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.507267 21170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.507668 21013 server_base.cc:1034] running on GCE node
I20260811 10:12:35.507934 21013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.508209 21013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.509383 21013 hybrid_clock.cc:648] HybridClock initialized: now 1786443155509370 us; error 37 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.512219 21013 init.cc:377] Logged in from keytab as oryx/127.8.238.1@KRBTEST.COM (short username oryx)
I20260811 10:12:35.512719 21013 webserver.cc:460] Webserver started at http://127.8.238.1:45991/ using document root <none> and password file <none>
I20260811 10:12:35.513007 21013 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.513090 21013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.514895 21013 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:35.515673 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.515887 21013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:35.516000 21013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "2092d73a6ea24855ab50b2d1d3c2d057"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "39eb396cf0489a252f5f77672859f36c"
server_key_iv: "15a6ef3443dce8f13ceaf35d4ca6d436"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.516383 21013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.537261 21013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.539786 21013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:35.540007 21013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.540372 21013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:35.540709 21013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:35.540766 21013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.540808 21013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:35.540833 21013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.552175 21013 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:36539
I20260811 10:12:35.552290 21385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:36539 every 8 connection(s)
I20260811 10:12:35.552644 21013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:35.553076 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 21013
I20260811 10:12:35.553242 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 20101
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
I20260811 10:12:35.568876 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:44321
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:43017
--webserver_port=36799
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
I20260811 10:12:35.572871 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.559919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:40545 (local address 127.8.238.61:37013)
0811 10:12:35.561028 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.561030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.561044 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:12:35.561079 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.561080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.561093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.561163 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.563815 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.564513 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565668 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.565966 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.566522 (+ 556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.569293 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.569295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.569295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.569311 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.571168 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.571704 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.571705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.571705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.571806 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572123 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.572124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.572124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.572280 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.572337 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.572686 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.572778 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073}
I20260811 10:12:35.573118 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.553523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.8.238.1:40545)
0811 10:12:35.556421 (+ 2898us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.556433 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.556581 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.563359 (+ 6778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.563360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.563526 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.563673 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.563676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.564606 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.564607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.565570 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.565572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.566318 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.566319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.566451 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.566875 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.566881 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.569144 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.571269 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.571277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.571280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.571605 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.571880 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.571881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.571881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.572013 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.572372 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.572380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.572540 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.572648 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.572823 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2847,"mutex_wait_us":32,"thread_start_us":89,"threads_started":1}
I20260811 10:12:35.573169 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.560059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:35253 (local address 127.8.238.60:42069)
0811 10:12:35.561129 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.561131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.561142 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:35.561171 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.561172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.561186 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.561251 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.562335 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.563440 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565108 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.565471 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565536 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.569656 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.569658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.569658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.569667 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.571671 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572093 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.572094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.572094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.572175 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572405 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.572407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.572407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.572601 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.572676 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.572912 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.573018 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":938,"threads_started":1}
I20260811 10:12:35.573611 21389 heartbeater.cc:347] Connected to a master server at 127.8.238.61:37013
I20260811 10:12:35.573720 21389 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.573769 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.559988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:41635 (local address 127.8.238.62:43017)
0811 10:12:35.561250 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.561253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.561265 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:12:35.561296 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.561297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.561310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.561373 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.563345 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.564172 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565285 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.565612 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565674 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.569948 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.569950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.569951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.569960 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.572318 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572750 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.572751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.572752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.572831 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.573048 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.573050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.573050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.573204 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.573269 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.573579 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.573678 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216}
I20260811 10:12:35.573916 21389 heartbeater.cc:511] Master 127.8.238.61:37013 requested a full tablet report, sending...
I20260811 10:12:35.574080 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.553524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:41635)
0811 10:12:35.553862 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.553906 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.555586 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.562866 (+ 7280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.562867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.563096 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.563231 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.563233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.564266 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.564267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.565186 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.565189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.565730 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.565801 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.566531 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.566535 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.569511 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.572441 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.572443 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.572665 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.572906 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.572906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.572990 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.573406 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.573406 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.573452 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.573877 (+ 425us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.573993 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":1,"thread_start_us":204,"threads_started":1}
I20260811 10:12:35.574404 21388 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:35.574450 21388 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.574507 21388 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:35.574504 20761 ts_manager.cc:194] Registered new tserver with Master: 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539)
I20260811 10:12:35.574777 19762 ts_manager.cc:194] Re-registered known tserver with Master: 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539)
I20260811 10:12:35.575230 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:41635
I20260811 10:12:35.575984 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.553957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.8.238.1:35253)
0811 10:12:35.556459 (+ 2502us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.556470 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.556581 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.561447 (+ 4866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.561452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.561898 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.562228 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.562235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.563565 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.563566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.565007 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.565011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.565620 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.565622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.565785 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.566456 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.566468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.569563 (+ 3095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.571778 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.571779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.571780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.571999 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.572248 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.572249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.572249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.572309 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.572706 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.572707 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.572764 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.575642 (+ 2878us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.575812 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2463,"mutex_wait_us":3,"thread_start_us":2333,"threads_started":1}
I20260811 10:12:35.576700 21386 heartbeater.cc:347] Connected to a master server at 127.8.238.60:42069
I20260811 10:12:35.576758 21386 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.576833 21386 heartbeater.cc:511] Master 127.8.238.60:42069 requested a full tablet report, sending...
I20260811 10:12:35.577173 21080 ts_manager.cc:194] Registered new tserver with Master: 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539)
W20260811 10:12:35.657593 21399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:35.657847 21399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:35.657889 21399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:35.659063 21399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:35.659129 21399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:35.659175 21399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:35.660995 21399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:44321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=36799
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:35.661248 21399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:35.661515 21399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.666872 21408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.666836 21409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.666849 21413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.666888 21399 server_base.cc:1034] running on GCE node
I20260811 10:12:35.667243 21399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.667528 21399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.668694 21399 hybrid_clock.cc:648] HybridClock initialized: now 1786443155668687 us; error 36 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.671258 21399 init.cc:377] Logged in from keytab as oryx/127.8.238.2@KRBTEST.COM (short username oryx)
I20260811 10:12:35.671705 21399 webserver.cc:460] Webserver started at http://127.8.238.2:36799/ using document root <none> and password file <none>
I20260811 10:12:35.671964 21399 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.672049 21399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.673405 21399 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:35.674043 21428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.674304 21399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:35.674422 21399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "cf56422974ea423085ae144260bca2e8"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "2e4f5290130a7be595c573416bd732d6"
server_key_iv: "72765a881bb7ac0f56297640f7ec83ac"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.674777 21399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.712215 21399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.714442 21399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:35.714630 21399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.714921 21399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:35.715279 21399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:35.715337 21399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.715379 21399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:35.715423 21399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.723244 21399 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:44321
I20260811 10:12:35.723322 21619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:44321 every 8 connection(s)
I20260811 10:12:35.723771 21399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:35.727008 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 21399
I20260811 10:12:35.727185 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 20330
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
I20260811 10:12:35.740703 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:37283 (local address 127.8.238.60:42069)
0811 10:12:35.724956 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.724959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.725092 (+ 133us) server_negotiation.cc:407] Connection header received
0811 10:12:35.725967 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.725969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.725987 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.726021 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.727098 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.727913 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.734304 (+ 6391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.734617 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.734660 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.737463 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.737465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.737465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.737474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.739086 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.739564 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.739566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.739566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.739618 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.739872 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.739873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.739873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.740021 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.740064 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.740453 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.740590 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:12:35.740875 21630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.8.238.2:37283)
0811 10:12:35.724940 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.724985 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.725830 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.726111 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.726114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.726704 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.726974 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.726981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.731940 (+ 4959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.731941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.734165 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.734169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.734751 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.734752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.734956 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.735549 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.735559 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.737348 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.739182 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.739189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.739191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.739490 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.739677 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.739678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.739678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.739810 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.740112 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.740118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.740277 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.740384 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.740555 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":116,"threads_started":1}
I20260811 10:12:35.741545 21622 heartbeater.cc:347] Connected to a master server at 127.8.238.60:42069
I20260811 10:12:35.741636 21622 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.741820 21622 heartbeater.cc:511] Master 127.8.238.60:42069 requested a full tablet report, sending...
I20260811 10:12:35.742381 21080 ts_manager.cc:194] Registered new tserver with Master: cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
I20260811 10:12:35.751475 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:54489 (local address 127.8.238.62:43017)
0811 10:12:35.724927 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.724930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.725103 (+ 173us) server_negotiation.cc:407] Connection header received
0811 10:12:35.725970 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.725972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.725986 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.726021 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.727098 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.727913 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.744930 (+ 17017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.745245 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.745302 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.748453 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.748455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.748456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.748465 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.750256 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.750641 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.750642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.750643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.750708 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.750923 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.750924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.750924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.751043 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.751117 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.751294 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.751379 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 10:12:35.751711 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:54489)
0811 10:12:35.725016 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.725027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.725830 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.726108 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.726113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.726704 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.726974 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.726981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.744159 (+ 17178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.744160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.744818 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.744821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.745363 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.745364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.745434 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.746510 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.746516 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.748344 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.750373 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.750374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.750375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.750559 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.750786 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.750787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.750787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.750850 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.751120 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.751121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.751167 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.751248 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.751371 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260811 10:12:35.752131 21623 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:35.752189 21623 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.752277 21623 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:35.752643 19762 ts_manager.cc:194] Re-registered known tserver with Master: cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
I20260811 10:12:35.753059 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:54489
I20260811 10:12:35.753955 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:33815
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:43017
--webserver_port=42807
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:12:35.755934 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:56815 (local address 127.8.238.61:37013)
0811 10:12:35.725031 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.725033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.725270 (+ 237us) server_negotiation.cc:407] Connection header received
0811 10:12:35.725942 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.725944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.725956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.726005 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.727219 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.747986 (+ 20767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.749109 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.749312 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.749406 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.751996 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.751998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.751998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.752006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.754529 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.754907 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.754908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.754908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.754969 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.755190 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.755191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.755191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.755310 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.755382 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.755710 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.755827 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:35.756331 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.724655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.8.238.2:56815)
0811 10:12:35.725199 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.725210 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.725841 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.726386 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.726387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.726672 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.726984 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.726988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.748123 (+ 21135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.748124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.749000 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.749003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.749405 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.749407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.749571 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.750057 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.750062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.751884 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.754656 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.754657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.754657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.754822 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.755060 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.755061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.755061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.755121 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.755387 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.755389 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.755513 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.756061 (+ 548us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.756199 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":28,"thread_start_us":165,"threads_started":1}
I20260811 10:12:35.757236 21624 heartbeater.cc:347] Connected to a master server at 127.8.238.61:37013
I20260811 10:12:35.757296 21624 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.757387 21624 heartbeater.cc:511] Master 127.8.238.61:37013 requested a full tablet report, sending...
I20260811 10:12:35.757687 20761 ts_manager.cc:194] Registered new tserver with Master: cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
I20260811 10:12:35.759104 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.755846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.0.0.1:39376)
0811 10:12:35.755937 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.755948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.755998 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.756301 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.756302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.756304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.756352 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.756354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.757052 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.757053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.758240 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.758244 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.758867 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.758868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.758932 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.758935 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.758981 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.759034 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 10:12:35.759102 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.756086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.8.238.60:42069)
0811 10:12:35.756128 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.756130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.756139 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:35.756164 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.756165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.756193 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.756252 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.756431 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.756955 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.758352 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.758742 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.758863 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.758978 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.758998 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.759037 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:35.759385 21098 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 10:12:35.759599 21150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6c28fcf12f4425d919b8ff54d0fd0b3. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.759711 21150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: LEARNER
W20260811 10:12:35.840977 21635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:35.841249 21635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:35.841288 21635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:35.842468 21635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:35.842533 21635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:35.842576 21635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:35.844101 21635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:33815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=42807
--tserver_master_addrs=127.8.238.62:43017,127.8.238.61:37013,127.8.238.60:42069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:35.844367 21635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:35.844606 21635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:35.849956 21643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.850057 21635 server_base.cc:1034] running on GCE node
W20260811 10:12:35.849891 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:35.849912 21644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:35.850399 21635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:35.850690 21635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:35.851866 21635 hybrid_clock.cc:648] HybridClock initialized: now 1786443155851859 us; error 37 us; skew 500 ppm
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:35.854367 21635 init.cc:377] Logged in from keytab as oryx/127.8.238.3@KRBTEST.COM (short username oryx)
I20260811 10:12:35.854830 21635 webserver.cc:460] Webserver started at http://127.8.238.3:42807/ using document root <none> and password file <none>
I20260811 10:12:35.855064 21635 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:35.855150 21635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:35.856659 21635 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:35.857271 21659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.857451 21635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:35.857563 21635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "1e4781dbf11449c492a52e214a205408"
format_stamp: "Formatted at 2026-08-11 10:12:34 on dist-test-slave-7f01"
server_key: "60e03913dfbccd67f5a646a19c889392"
server_key_iv: "6a26a39ab2c80c933ad775c0e5ac3cdb"
server_key_version: "encryptionkey@0"
I20260811 10:12:35.857913 21635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:35.878693 21635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:35.882246 21635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:35.882423 21635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:35.882691 21635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:35.883100 21635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:35.883148 21635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.883183 21635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:35.883210 21635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:35.890527 21635 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33815
I20260811 10:12:35.890573 21845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33815 every 8 connection(s)
I20260811 10:12:35.890925 21635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.61@KRBTEST.COM
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
Aug 11 10:12:35 dist-test-slave-7f01 krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443155, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.60@KRBTEST.COM
I20260811 10:12:35.900995 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 21635
I20260811 10:12:35.902137 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.891935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:38615 (local address 127.8.238.62:43017)
0811 10:12:35.891990 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.891992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.892195 (+ 203us) server_negotiation.cc:407] Connection header received
0811 10:12:35.892755 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.892757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.892773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.892829 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.893757 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894349 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.895381 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.895598 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.895680 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.898810 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.898812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.898813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.898821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.900683 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901183 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.901185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.901185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.901252 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901498 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.901499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.901499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.901639 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.901685 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.901914 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.902038 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:12:35.902137 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.891792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:44613 (local address 127.8.238.61:37013)
0811 10:12:35.891885 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.891887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.892213 (+ 326us) server_negotiation.cc:407] Connection header received
0811 10:12:35.892766 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.892768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.892781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.892817 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.893768 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894669 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.895899 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.896171 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.896265 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.898454 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.898455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.898456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.898465 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.900310 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.900904 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.900906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.900906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.900981 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901341 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.901342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.901342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.901491 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.901536 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.901914 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.902014 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 10:12:35.902734 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.891785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:60401 (local address 127.8.238.60:42069)
0811 10:12:35.891845 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.891847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.892249 (+ 402us) server_negotiation.cc:407] Connection header received
0811 10:12:35.892765 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.892766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.892787 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.892819 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.893768 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894678 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.896325 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.896641 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.896731 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.899454 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.899456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.899456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.899464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.901399 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901859 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.901860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.901861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.901921 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.902129 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.902130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.902130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.902268 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.902319 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.902562 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.902643 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:12:35.902902 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.891872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:38615)
0811 10:12:35.892081 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.892094 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.892641 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.892968 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.892983 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.893428 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.893653 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.893658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894466 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.894467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.895280 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.895282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.895689 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.895690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.895880 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.896530 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.896541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.898679 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.900839 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.900840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.900868 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.901087 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.901331 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.901333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.901405 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.901737 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.901739 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.901782 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.902413 (+ 631us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.902617 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260811 10:12:35.903348 21847 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43017
I20260811 10:12:35.903478 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.892032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.8.238.3:60401)
0811 10:12:35.892182 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.892192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.892663 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.892975 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.892980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.893425 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.893655 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.893661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894786 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.894787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.895808 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.895811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.896732 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.896733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.896810 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.897253 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.897258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.899366 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.901534 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.901535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.901766 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.901999 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.902000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.902000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.902059 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.902375 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.902376 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.902436 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.902862 (+ 426us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.903301 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":413,"thread_start_us":85,"threads_started":1}
I20260811 10:12:35.903488 21847 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.903875 21846 heartbeater.cc:347] Connected to a master server at 127.8.238.60:42069
I20260811 10:12:35.903935 21846 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.904031 21846 heartbeater.cc:511] Master 127.8.238.60:42069 requested a full tablet report, sending...
I20260811 10:12:35.904134 21851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.891802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.8.238.3:44613)
0811 10:12:35.892048 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.892084 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.892663 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.892975 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.892980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.893426 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:35.893654 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.893661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.894786 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.894787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.895808 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.895811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.896269 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.896270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.896336 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.896707 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:35.896713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:35.898356 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:35.900408 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.900417 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.900419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.900804 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.901061 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:35.901061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:35.901062 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:35.901252 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:35.901583 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:35.901591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:35.901758 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.902715 (+ 957us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.903242 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":53,"thread_start_us":93,"threads_started":1}
I20260811 10:12:35.904028 21847 heartbeater.cc:511] Master 127.8.238.62:43017 requested a full tablet report, sending...
I20260811 10:12:35.904604 21080 ts_manager.cc:194] Registered new tserver with Master: 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:35.904713 21848 heartbeater.cc:347] Connected to a master server at 127.8.238.61:37013
I20260811 10:12:35.904755 21848 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:35.904824 21848 heartbeater.cc:511] Master 127.8.238.61:37013 requested a full tablet report, sending...
I20260811 10:12:35.905124 20761 ts_manager.cc:194] Registered new tserver with Master: 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:35.905198 19762 ts_manager.cc:194] Re-registered known tserver with Master: 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:35.905696 19762 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:38615
I20260811 10:12:35.911300 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.903455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.8.238.61:37013)
0811 10:12:35.903492 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.903494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.904649 (+ 1155us) server_negotiation.cc:407] Connection header received
0811 10:12:35.904745 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.904746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.904756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.904786 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.905679 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.906173 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.906869 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.907069 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.907147 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.908451 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.908453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.908453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.908460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.910085 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.910419 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.910420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.910420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.910484 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.910703 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.910704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.910704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.910844 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.910948 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.911119 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.911200 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:12:35.912251 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.903099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.8.238.60:42069)
0811 10:12:35.903135 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.903136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.904510 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 10:12:35.904564 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.904565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.904575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.904622 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.905431 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.906072 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.907126 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.907412 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.907463 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.909135 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.909136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.909137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.909144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.910877 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.911330 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.911331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.911331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.911389 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.911643 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.911644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.911644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.911814 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.911932 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.912066 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.912134 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:12:35.912952 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.902418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.8.238.62:43017)
0811 10:12:35.902458 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.902460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.902727 (+ 267us) server_negotiation.cc:407] Connection header received
0811 10:12:35.902821 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.902822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.902833 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.902862 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.904214 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.904938 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.907321 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.908223 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.908270 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.909736 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.909737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.909738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.909745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.911672 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.912088 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.912090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.912090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.912147 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.912399 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.912400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.912400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.912525 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.912580 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.912758 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.912836 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:12:35.914507 19762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:35.916278 19762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:35.916982 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: NON_VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: true } } }
I20260811 10:12:35.917083 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.918103 21876 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: attempting to promote NON_VOTER 32d69584b4f14f15ba0b037aa1d3078d to VOTER
I20260811 10:12:35.918251 21876 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:35.918803 21098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 LEARNER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 10:12:35.918830 20778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Refusing update from remote peer b6c28fcf12f4425d919b8ff54d0fd0b3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 10:12:35.919006 21027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 10:12:35.919128 21874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 10:12:35.919921 21027 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.920076 20778 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.920082 21098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.920928 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.921010 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.921075 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.921132 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.921168 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.921219 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:35.921161 21150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.921250 21150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:35.921686 21896 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:35.921783 21896 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:35.927373 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:44321 (local address 127.0.0.1:53426)
0811 10:12:35.923413 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.923425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.923466 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.923970 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.923972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.923974 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.924021 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.924023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925164 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.925165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.926293 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.926296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.927151 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.927212 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.927215 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.927262 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.927301 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":1}
I20260811 10:12:35.927815 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.8.238.2:44321)
0811 10:12:35.923785 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.923789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.923801 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:12:35.923850 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.923852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.923871 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.923958 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.924086 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925073 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.926398 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.927045 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927143 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.927355 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.927387 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.927435 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260811 10:12:35.927983 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36539 (local address 127.0.0.1:46888)
0811 10:12:35.923636 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.923647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.923702 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.924062 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.924063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.924065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.924104 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.924105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925426 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.925427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.926508 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.926510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.927702 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.927795 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.927799 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.927841 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.927889 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":103,"threads_started":1}
I20260811 10:12:35.928011 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.8.238.3:33815)
0811 10:12:35.923645 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.923652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.923906 (+ 254us) server_negotiation.cc:407] Connection header received
0811 10:12:35.924029 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.924031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.924046 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.924095 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.924308 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925363 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.926283 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.926943 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927011 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.927084 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.927789 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.927859 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260811 10:12:35.928218 21897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33815 (local address 127.0.0.1:46058)
0811 10:12:35.923830 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.923842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.923936 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.924180 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.924181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.924183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.924225 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.924227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925455 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.925456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.926185 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.926187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.927064 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.927162 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.927170 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.928085 (+ 915us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.928136 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":72,"threads_started":1}
I20260811 10:12:35.928398 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.923551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.8.238.1:36539)
0811 10:12:35.923816 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.923821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.923830 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:35.923877 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.923878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.923891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.923990 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.924163 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.925334 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.926644 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.927597 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.927692 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.928008 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.928266 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.928314 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260811 10:12:35.928670 21519 tablet_service.cc:1467] Processing CreateTablet for tablet 945a51f95a894df4bc5d14f1dff89710 (DEFAULT_TABLE table=test-table [id=ae7dec3b7af84033941e33d919adc17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:35.929088 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945a51f95a894df4bc5d14f1dff89710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:35.929018 21738 tablet_service.cc:1467] Processing CreateTablet for tablet 945a51f95a894df4bc5d14f1dff89710 (DEFAULT_TABLE table=test-table [id=ae7dec3b7af84033941e33d919adc17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:35.929121 21271 tablet_service.cc:1467] Processing CreateTablet for tablet 945a51f95a894df4bc5d14f1dff89710 (DEFAULT_TABLE table=test-table [id=ae7dec3b7af84033941e33d919adc17a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:35.929543 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945a51f95a894df4bc5d14f1dff89710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:35.929584 21271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 945a51f95a894df4bc5d14f1dff89710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:35.932988 21913 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Bootstrap starting.
I20260811 10:12:35.934137 21916 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Bootstrap starting.
I20260811 10:12:35.934576 21913 tablet_bootstrap.cc:654] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:35.935348 21916 tablet_bootstrap.cc:654] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:35.935398 21913 log.cc:824] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:35.935999 21917 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Bootstrap starting.
I20260811 10:12:35.936317 21916 log.cc:824] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:35.936784 21917 tablet_bootstrap.cc:654] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:35.936987 21913 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: No bootstrap required, opened a new log
I20260811 10:12:35.937064 21913 ts_tablet_manager.cc:1397] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 10:12:35.937362 21916 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: No bootstrap required, opened a new log
I20260811 10:12:35.937415 21917 log.cc:824] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:35.937439 21916 ts_tablet_manager.cc:1397] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:12:35.938742 21917 tablet_bootstrap.cc:492] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: No bootstrap required, opened a new log
I20260811 10:12:35.938804 21917 ts_tablet_manager.cc:1397] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:12:35.939105 21913 raft_consensus.cc:348] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.939231 21913 raft_consensus.cc:374] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:35.939253 21913 raft_consensus.cc:729] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf56422974ea423085ae144260bca2e8, State: Initialized, Role: FOLLOWER
I20260811 10:12:35.939375 21913 consensus_queue.cc:260] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.939680 21913 ts_tablet_manager.cc:1428] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:35.939844 21623 heartbeater.cc:503] Master 127.8.238.62:43017 was elected leader, sending a full tablet report...
I20260811 10:12:35.941825 21916 raft_consensus.cc:348] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.941987 21916 raft_consensus.cc:374] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:35.942025 21916 raft_consensus.cc:729] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4781dbf11449c492a52e214a205408, State: Initialized, Role: FOLLOWER
I20260811 10:12:35.942135 21916 consensus_queue.cc:260] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.942480 21916 ts_tablet_manager.cc:1428] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 10:12:35.942560 21847 heartbeater.cc:503] Master 127.8.238.62:43017 was elected leader, sending a full tablet report...
I20260811 10:12:35.944185 21917 raft_consensus.cc:348] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.944305 21917 raft_consensus.cc:374] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:35.944325 21917 raft_consensus.cc:729] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2092d73a6ea24855ab50b2d1d3c2d057, State: Initialized, Role: FOLLOWER
I20260811 10:12:35.944454 21917 consensus_queue.cc:260] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.944810 21917 ts_tablet_manager.cc:1428] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260811 10:12:35.944854 21388 heartbeater.cc:503] Master 127.8.238.62:43017 was elected leader, sending a full tablet report...
I20260811 10:12:35.945080 21926 raft_consensus.cc:482] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:35.945230 21926 raft_consensus.cc:504] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.945616 21926 leader_election.cc:288] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539), 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:35.948758 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.945858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:58981 (local address 127.8.238.3:33815)
0811 10:12:35.945900 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.945902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.945950 (+ 48us) server_negotiation.cc:407] Connection header received
0811 10:12:35.946020 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.946021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.946032 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.946087 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.946391 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.946874 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948018 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.948398 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948492 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.948502 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.948653 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.948698 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:12:35.948760 21628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.945800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33815 (local address 127.8.238.2:58981)
0811 10:12:35.945901 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.945913 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.945964 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.946088 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.946090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.946091 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.946294 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.946296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.946962 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.946964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.947933 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.947936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.948491 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.948581 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.948584 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.948626 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.948662 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":1}
I20260811 10:12:35.948833 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.945985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:45003 (local address 127.8.238.1:36539)
0811 10:12:35.946033 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.946034 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.946040 (+ 6us) server_negotiation.cc:407] Connection header received
0811 10:12:35.946059 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.946060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.946070 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.946115 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:35.946407 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.946874 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948038 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.948393 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948482 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.948493 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.948729 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.948766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:12:35.948975 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.945771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36539 (local address 127.8.238.2:45003)
0811 10:12:35.945859 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:35.945870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:35.945912 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:35.946171 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:35.946173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:35.946174 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:35.946327 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.946329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.946977 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.946979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.947936 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:35.947939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.948483 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.948484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:35.948643 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.948647 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:35.948706 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:12:35.948746 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":5}
I20260811 10:12:35.949211 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945a51f95a894df4bc5d14f1dff89710" candidate_uuid: "cf56422974ea423085ae144260bca2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4781dbf11449c492a52e214a205408" is_pre_election: true
I20260811 10:12:35.949338 21769 raft_consensus.cc:2454] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf56422974ea423085ae144260bca2e8 in term 0.
I20260811 10:12:35.949436 21307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945a51f95a894df4bc5d14f1dff89710" candidate_uuid: "cf56422974ea423085ae144260bca2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" is_pre_election: true
I20260811 10:12:35.949587 21307 raft_consensus.cc:2454] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf56422974ea423085ae144260bca2e8 in term 0.
I20260811 10:12:35.949878 21435 leader_election.cc:302] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2092d73a6ea24855ab50b2d1d3c2d057, cf56422974ea423085ae144260bca2e8; no voters:
I20260811 10:12:35.950110 21926 raft_consensus.cc:2781] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:35.950176 21926 raft_consensus.cc:482] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:35.950199 21926 raft_consensus.cc:3037] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:35.951130 21926 raft_consensus.cc:504] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.951288 21926 leader_election.cc:288] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [CANDIDATE]: Term 1 election: Requested vote from peers 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539), 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815)
I20260811 10:12:35.951444 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945a51f95a894df4bc5d14f1dff89710" candidate_uuid: "cf56422974ea423085ae144260bca2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4781dbf11449c492a52e214a205408"
I20260811 10:12:35.951550 21769 raft_consensus.cc:3037] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:35.951535 21307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "945a51f95a894df4bc5d14f1dff89710" candidate_uuid: "cf56422974ea423085ae144260bca2e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2092d73a6ea24855ab50b2d1d3c2d057"
I20260811 10:12:35.951622 21307 raft_consensus.cc:3037] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:35.952596 21307 raft_consensus.cc:2454] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf56422974ea423085ae144260bca2e8 in term 1.
I20260811 10:12:35.952739 21769 raft_consensus.cc:2454] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf56422974ea423085ae144260bca2e8 in term 1.
I20260811 10:12:35.952765 21435 leader_election.cc:302] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2092d73a6ea24855ab50b2d1d3c2d057, cf56422974ea423085ae144260bca2e8; no voters:
I20260811 10:12:35.952844 21926 raft_consensus.cc:2781] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:35.952991 21926 raft_consensus.cc:686] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 1 LEADER]: Becoming Leader. State: Replica: cf56422974ea423085ae144260bca2e8, State: Running, Role: LEADER
I20260811 10:12:35.953111 21926 consensus_queue.cc:237] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } }
I20260811 10:12:35.953876 19762 catalog_manager.cc:5495] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf56422974ea423085ae144260bca2e8 (127.8.238.2). New cstate: current_term: 1 leader_uuid: "cf56422974ea423085ae144260bca2e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:35.962143 21908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.959714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53438 (local address 127.8.238.2:44321)
0811 10:12:35.959778 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.959781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.959790 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:35.959869 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.959870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.959891 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.959955 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:35.960163 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.960668 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.961431 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.961618 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.961685 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.961860 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:35.961959 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:35.962022 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.962036 (+ 14us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.962051 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:12:35.963936 21307 raft_consensus.cc:1261] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Refusing update from remote peer cf56422974ea423085ae144260bca2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:35.963936 21769 raft_consensus.cc:1261] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Refusing update from remote peer cf56422974ea423085ae144260bca2e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:35.964288 21926 consensus_queue.cc:1035] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:35.964453 21934 consensus_queue.cc:1035] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:35.969071 21940 mvcc.cc:205] Tried to move back new op lower bound from 7317271166825590784 to 7317271166784647168. Current Snapshot: MvccSnapshot[applied={T|T < 7317271166825590784 or (T in {7317271166825590784})}]
I20260811 10:12:35.969245 21941 mvcc.cc:205] Tried to move back new op lower bound from 7317271166825590784 to 7317271166784647168. Current Snapshot: MvccSnapshot[applied={T|T < 7317271166825590784}]
I20260811 10:12:35.970649 19762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:35.970752 19762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:35.972185 19762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6c28fcf12f4425d919b8ff54d0fd0b3: Sending DeleteTablet for 3 replicas of tablet 945a51f95a894df4bc5d14f1dff89710
I20260811 10:12:35.972522 21271 tablet_service.cc:1514] Processing DeleteTablet for tablet 945a51f95a894df4bc5d14f1dff89710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:35 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:46888
I20260811 10:12:35.972630 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet 945a51f95a894df4bc5d14f1dff89710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:35 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:46058
I20260811 10:12:35.973290 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 19712
I20260811 10:12:35.973397 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet 945a51f95a894df4bc5d14f1dff89710 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:35 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:53426
I20260811 10:12:35.983043 21951 tablet_replica.cc:330] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: stopping tablet replica
I20260811 10:12:35.983183 21951 raft_consensus.cc:2229] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:35.983296 21951 raft_consensus.cc:2258] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:35.983522 21956 tablet_replica.cc:330] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: stopping tablet replica
I20260811 10:12:35.983616 21956 raft_consensus.cc:2229] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:35.983712 21956 raft_consensus.cc:2258] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:35.984376 21951 ts_tablet_manager.cc:1905] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:35.986457 21951 ts_tablet_manager.cc:1918] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:35.986552 21951 log.cc:1196] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/945a51f95a894df4bc5d14f1dff89710
I20260811 10:12:35.986897 21951 ts_tablet_manager.cc:1939] T 945a51f95a894df4bc5d14f1dff89710 P 2092d73a6ea24855ab50b2d1d3c2d057: Deleting consensus metadata
I20260811 10:12:35.987689 21956 ts_tablet_manager.cc:1905] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:35.989622 21956 ts_tablet_manager.cc:1918] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:35.989709 21956 log.cc:1196] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/945a51f95a894df4bc5d14f1dff89710
I20260811 10:12:35.990016 21956 ts_tablet_manager.cc:1939] T 945a51f95a894df4bc5d14f1dff89710 P cf56422974ea423085ae144260bca2e8: Deleting consensus metadata
I20260811 10:12:35.990305 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.973099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.8.238.60:42069)
0811 10:12:35.973142 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.973144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.982046 (+ 8902us) server_negotiation.cc:407] Connection header received
0811 10:12:35.982154 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.982155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.982167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.982198 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.982946 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.983693 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.984756 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.985068 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.985115 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.987027 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.987028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.987029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.987038 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:35.988851 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.989291 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.989293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.989293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.989371 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:35.989609 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:35.989610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:35.989610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:35.989767 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:35.989851 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:35.990100 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 10:12:35.990218 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 10:12:35.990698 21189 connection.cc:535] server connection from 127.0.0.1:46888 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:12:35.990692 21434 connection.cc:535] server connection from 127.0.0.1:53426 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:12:35.991298 20734 connection.cc:535] server connection from 127.0.0.1:56758 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:12:35.972901 21952 tablet_replica.cc:330] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: stopping tablet replica
I20260811 10:12:35.991652 21952 raft_consensus.cc:2229] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:35.991802 21952 raft_consensus.cc:2258] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:35.992471 21952 ts_tablet_manager.cc:1905] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:35.994361 21952 ts_tablet_manager.cc:1918] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:35.994477 21952 log.cc:1196] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/945a51f95a894df4bc5d14f1dff89710
I20260811 10:12:35.994807 21952 ts_tablet_manager.cc:1939] T 945a51f95a894df4bc5d14f1dff89710 P 1e4781dbf11449c492a52e214a205408: Deleting consensus metadata
W20260811 10:12:35.995280 21660 connection.cc:441] server connection from 127.0.0.1:46058 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46058 (request call id 1) could send its response
I20260811 10:12:36.001358 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.993746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.8.238.61:37013)
0811 10:12:35.993794 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.993796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.993804 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:35.993830 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.993831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.993843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.993892 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.995187 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.995804 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.996525 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.996736 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.996816 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.998317 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.998319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.998319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.998327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:36.000050 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:36.000467 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:36.000469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:36.000469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:36.000530 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:36.000743 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:36.000744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:36.000745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:36.000878 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:36.000949 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:36.001189 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 10:12:36.001274 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:36.002317 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:35.993692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.8.238.60:42069)
0811 10:12:35.993740 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:35.993742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:35.993907 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:12:35.994045 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:35.994046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:35.994057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:35.994086 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:35.995058 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.995836 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.996935 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:35.997566 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:35.997645 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:35.999000 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:35.999002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:35.999002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:35.999009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:36.000744 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:36.001112 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:36.001113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:36.001113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:36.001173 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:36.001768 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:36.001769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:36.001770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:36.001914 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:36.001991 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:36.002186 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 10:12:36.002257 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:36.969771 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:36.969300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:40519)
0811 10:12:36.969532 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:36.969633 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":112,"threads_started":1}
I20260811 10:12:36.969765 21967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:36.969488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:42037)
0811 10:12:36.969612 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:36.969701 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
W20260811 10:12:36.970034 21388 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:43017 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
W20260811 10:12:36.970119 21623 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:43017 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:36.971501 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:36.971229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:44401)
0811 10:12:36.971378 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:36.971452 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":46,"threads_started":1}
W20260811 10:12:36.971725 21847 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:43017 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:37.473101 21972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b6c28fcf12f4425d919b8ff54d0fd0b3)
I20260811 10:12:37.473191 21973 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b6c28fcf12f4425d919b8ff54d0fd0b3)
I20260811 10:12:37.473331 21973 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:37.473332 21972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:37.473664 21973 leader_election.cc:288] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60:42069)
I20260811 10:12:37.473881 21972 leader_election.cc:288] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61:37013)
I20260811 10:12:37.474092 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.473752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34042)
0811 10:12:37.473989 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474043 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":67,"threads_started":1}
I20260811 10:12:37.474303 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.473732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34036)
0811 10:12:37.474198 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474258 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":61,"threads_started":1}
W20260811 10:12:37.474390 20734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
W20260811 10:12:37.474418 21059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:37.474979 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.474894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34060)
0811 10:12:37.474931 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474955 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 10:12:37.474980 21975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.474894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34058)
0811 10:12:37.474932 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474955 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 10:12:37.475046 21059 leader_election.cc:334] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
W20260811 10:12:37.475219 20734 leader_election.cc:334] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:37.477469 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.474102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.8.238.60:42069)
0811 10:12:37.474214 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:37.474217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:37.474225 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:37.474264 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:37.474265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:37.474277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:37.474350 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:37.474566 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.475358 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.476533 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.477022 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.477115 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:37.477129 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:37.477367 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 10:12:37.477412 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260811 10:12:37.477509 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.473824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:42069 (local address 127.0.0.1:59014)
0811 10:12:37.474006 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:37.474079 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:37.474412 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:37.474413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:37.474416 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:37.474478 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:37.474480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.475446 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.475447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:37.476412 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:37.476414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.477120 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.477121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:37.477270 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:37.477276 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:12:37.477342 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 10:12:37.477392 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":42,"threads_started":1}
I20260811 10:12:37.477675 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.474016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:37013 (local address 127.0.0.1:44512)
0811 10:12:37.474397 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.474493 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:37.474643 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:37.474897 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:37.474898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:37.474901 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:37.474940 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:37.474942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.475766 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.475767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:37.476663 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:37.476665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.477390 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.477391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:37.477479 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:37.477484 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:37.477541 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:12:37.477596 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":43,"threads_started":1}
I20260811 10:12:37.477733 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.474266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44512 (local address 127.8.238.61:37013)
0811 10:12:37.474725 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:37.474727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:37.474735 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:37.474766 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:37.474767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:37.474778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:37.474851 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:37.475022 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.475659 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.476762 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:37.477244 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:37.477372 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:37.477385 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:37.477617 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 10:12:37.477671 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":38,"threads_started":1}
I20260811 10:12:37.477828 21098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be7a19682dfc40eb99d4093b863a61e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32d69584b4f14f15ba0b037aa1d3078d" is_pre_election: true
I20260811 10:12:37.477945 21098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7a19682dfc40eb99d4093b863a61e3 in term 1.
I20260811 10:12:37.477905 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d69584b4f14f15ba0b037aa1d3078d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "be7a19682dfc40eb99d4093b863a61e3" is_pre_election: true
I20260811 10:12:37.478032 20778 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d69584b4f14f15ba0b037aa1d3078d in term 1.
I20260811 10:12:37.478120 20737 leader_election.cc:302] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d69584b4f14f15ba0b037aa1d3078d, be7a19682dfc40eb99d4093b863a61e3; no voters: b6c28fcf12f4425d919b8ff54d0fd0b3
I20260811 10:12:37.478219 21973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 10:12:37.478266 21973 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b6c28fcf12f4425d919b8ff54d0fd0b3)
I20260811 10:12:37.478291 21973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:12:37.478300 21061 leader_election.cc:302] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d69584b4f14f15ba0b037aa1d3078d, be7a19682dfc40eb99d4093b863a61e3; no voters: b6c28fcf12f4425d919b8ff54d0fd0b3
I20260811 10:12:37.478401 21972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 10:12:37.478433 21972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Starting leader election (detected failure of leader b6c28fcf12f4425d919b8ff54d0fd0b3)
I20260811 10:12:37.478458 21972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:12:37.479084 21973 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:37.479229 21973 leader_election.cc:288] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 election: Requested vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60:42069)
I20260811 10:12:37.479249 21972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:37.479393 21972 leader_election.cc:288] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 election: Requested vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), be7a19682dfc40eb99d4093b863a61e3 (127.8.238.61:37013)
I20260811 10:12:37.479401 21098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be7a19682dfc40eb99d4093b863a61e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32d69584b4f14f15ba0b037aa1d3078d"
I20260811 10:12:37.479494 21098 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate be7a19682dfc40eb99d4093b863a61e3 in current term 2: Already voted for candidate 32d69584b4f14f15ba0b037aa1d3078d in this term.
I20260811 10:12:37.479574 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.479474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34098)
0811 10:12:37.479517 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.479542 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 10:12:37.479588 20778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "32d69584b4f14f15ba0b037aa1d3078d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "be7a19682dfc40eb99d4093b863a61e3"
I20260811 10:12:37.479686 20778 raft_consensus.cc:2379] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 32d69584b4f14f15ba0b037aa1d3078d in current term 2: Already voted for candidate be7a19682dfc40eb99d4093b863a61e3 in this term.
I20260811 10:12:37.479799 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.479298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34084)
0811 10:12:37.479546 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.479586 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260811 10:12:37.479936 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.479814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34102)
0811 10:12:37.479875 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.479907 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 10:12:37.480031 21059 leader_election.cc:334] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:37.480106 21059 leader_election.cc:302] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 32d69584b4f14f15ba0b037aa1d3078d; no voters: b6c28fcf12f4425d919b8ff54d0fd0b3, be7a19682dfc40eb99d4093b863a61e3
I20260811 10:12:37.480183 21972 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 10:12:37.480347 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.479976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34114)
0811 10:12:37.480294 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.480330 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302}
W20260811 10:12:37.480420 20734 leader_election.cc:334] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:37.480468 20734 leader_election.cc:302] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be7a19682dfc40eb99d4093b863a61e3; no voters: 32d69584b4f14f15ba0b037aa1d3078d, b6c28fcf12f4425d919b8ff54d0fd0b3
I20260811 10:12:37.480543 21973 raft_consensus.cc:2726] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 10:12:37.970834 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.970583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:41675)
0811 10:12:37.970733 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.970772 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260811 10:12:37.970834 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.970583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:50903)
0811 10:12:37.970733 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.970772 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260811 10:12:37.972316 21989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:37.972087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:58887)
0811 10:12:37.972219 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:37.972261 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260811 10:12:38.971705 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:38.971445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:33583)
0811 10:12:38.971597 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:38.971641 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260811 10:12:38.971877 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:38.971522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:33373)
0811 10:12:38.971664 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:38.971705 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260811 10:12:38.972885 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:38.972691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:43793)
0811 10:12:38.972810 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:38.972848 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":84,"threads_started":1}
I20260811 10:12:39.121650 21997 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:39.121798 21997 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:39.122048 21997 leader_election.cc:288] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60:42069)
I20260811 10:12:39.122504 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.122225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34358)
0811 10:12:39.122398 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.122454 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":83,"threads_started":1}
I20260811 10:12:39.122416 21098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be7a19682dfc40eb99d4093b863a61e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32d69584b4f14f15ba0b037aa1d3078d" is_pre_election: true
I20260811 10:12:39.122566 21098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be7a19682dfc40eb99d4093b863a61e3 in term 2.
I20260811 10:12:39.122751 20737 leader_election.cc:302] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32d69584b4f14f15ba0b037aa1d3078d, be7a19682dfc40eb99d4093b863a61e3; no voters:
I20260811 10:12:39.122901 21997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 10:12:39.122960 21997 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:39.122983 21997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:12:39.123015 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.122889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34364)
0811 10:12:39.122944 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.122979 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 10:12:39.123114 20734 leader_election.cc:334] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:39.123910 21997 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:39.124066 21997 leader_election.cc:288] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 election: Requested vote from peers b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017), 32d69584b4f14f15ba0b037aa1d3078d (127.8.238.60:42069)
I20260811 10:12:39.124246 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.124154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34370)
0811 10:12:39.124192 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.124223 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 10:12:39.124249 21098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be7a19682dfc40eb99d4093b863a61e3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "32d69584b4f14f15ba0b037aa1d3078d"
I20260811 10:12:39.124326 21098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:12:39.124665 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.124540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34372)
0811 10:12:39.124614 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.124643 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 10:12:39.124773 20734 leader_election.cc:334] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
I20260811 10:12:39.125133 21098 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be7a19682dfc40eb99d4093b863a61e3 in term 3.
I20260811 10:12:39.125334 20737 leader_election.cc:302] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 32d69584b4f14f15ba0b037aa1d3078d, be7a19682dfc40eb99d4093b863a61e3; no voters: b6c28fcf12f4425d919b8ff54d0fd0b3
I20260811 10:12:39.125425 21997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 10:12:39.125530 21997 raft_consensus.cc:686] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [term 3 LEADER]: Becoming Leader. State: Replica: be7a19682dfc40eb99d4093b863a61e3, State: Running, Role: LEADER
I20260811 10:12:39.125592 21997 consensus_queue.cc:237] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } }
I20260811 10:12:39.125957 22004 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be7a19682dfc40eb99d4093b863a61e3. Latest consensus state: current_term: 3 leader_uuid: "be7a19682dfc40eb99d4093b863a61e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:39.126066 22004 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:39.126277 22007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:39.126823 22007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:39.126924 22007 catalog_manager.cc:1259] Loaded cluster ID: 925ee9ecc2c34d71bcb6c262c9cc4df7
I20260811 10:12:39.126959 22007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:39.127148 22007 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:39.127241 22007 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Loaded TSK: 0
I20260811 10:12:39.127398 22007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:12:39.151906 20759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:39.153726 20759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:39.155361 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.155231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34394)
0811 10:12:39.155296 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.155327 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 10:12:39.155424 21098 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [term 3 FOLLOWER]: Refusing update from remote peer be7a19682dfc40eb99d4093b863a61e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 10:12:39.155727 22004 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 10:12:39.155835 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.155707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.0.0.1:34408)
0811 10:12:39.155777 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.155814 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260811 10:12:39.155994 20734 consensus_peers.cc:487] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 -> Peer b6c28fcf12f4425d919b8ff54d0fd0b3 (127.8.238.62:43017): Couldn't send request to peer b6c28fcf12f4425d919b8ff54d0fd0b3. Status: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 10:12:39.156499 22013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: New leader be7a19682dfc40eb99d4093b863a61e3. Latest consensus state: current_term: 3 leader_uuid: "be7a19682dfc40eb99d4093b863a61e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:39.156633 22013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:39.157059 21997 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "be7a19682dfc40eb99d4093b863a61e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:39.157230 21997 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:39.157559 22013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "be7a19682dfc40eb99d4093b863a61e3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b6c28fcf12f4425d919b8ff54d0fd0b3" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43017 } } peers { permanent_uuid: "be7a19682dfc40eb99d4093b863a61e3" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 37013 } attrs { promote: false } } peers { permanent_uuid: "32d69584b4f14f15ba0b037aa1d3078d" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 42069 } attrs { promote: false } } }
I20260811 10:12:39.157657 22013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32d69584b4f14f15ba0b037aa1d3078d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:12:39.162460 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.159047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:44321 (local address 127.0.0.1:55576)
0811 10:12:39.159352 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.159361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:39.159395 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:39.159622 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:39.159623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:39.159626 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:39.159673 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.159674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160413 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.160414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.161548 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.161551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.162233 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.162234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.162293 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.162298 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:39.162354 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:12:39.162391 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":83,"threads_started":1}
I20260811 10:12:39.162526 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.159163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.8.238.2:44321)
0811 10:12:39.159487 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.159489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.159496 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:12:39.159524 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.159525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.159536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.159598 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:39.159776 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160310 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.161620 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.162142 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.162216 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.162230 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.162380 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.162434 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":66,"threads_started":1}
I20260811 10:12:39.162714 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.159005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36539 (local address 127.0.0.1:38272)
0811 10:12:39.159218 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.159227 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:39.159275 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:39.159540 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:39.159540 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:39.159542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:39.159579 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.159580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160708 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.160709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.161751 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.161754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.162516 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.162517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.162583 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.162588 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:39.162620 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 10:12:39.162661 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260811 10:12:39.162870 21519 tablet_service.cc:1467] Processing CreateTablet for tablet a65149e561d9430fab95f8c66beb92f2 (DEFAULT_TABLE table=test-table [id=9947891f439e40ef96e584cf7f6b839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:39.163023 21519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65149e561d9430fab95f8c66beb92f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:39.163112 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.159083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.8.238.1:36539)
0811 10:12:39.159309 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.159311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.159320 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:39.159426 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.159427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.159443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.159645 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:39.159690 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160628 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.161830 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.162413 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.162925 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.162957 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.162978 (+ 21us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.163020 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":54,"threads_started":1}
I20260811 10:12:39.163403 21271 tablet_service.cc:1467] Processing CreateTablet for tablet a65149e561d9430fab95f8c66beb92f2 (DEFAULT_TABLE table=test-table [id=9947891f439e40ef96e584cf7f6b839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:39.163468 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.158911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33815 (local address 127.0.0.1:48950)
0811 10:12:39.158964 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.158975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:39.159026 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:39.159474 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:39.159476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:39.159478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:39.159540 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.159542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160613 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.160615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.162482 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.162484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.163284 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.163285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.163340 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.163345 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:39.163373 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 10:12:39.163410 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":1008}
I20260811 10:12:39.163558 21271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65149e561d9430fab95f8c66beb92f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:39.163820 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.159192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.8.238.3:33815)
0811 10:12:39.159291 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.159294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.159303 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:39.159337 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.159337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.159349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.159635 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:39.159684 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.160520 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.162579 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.163182 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.163635 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.163650 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.163670 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.163709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":51,"threads_started":1}
I20260811 10:12:39.164139 21738 tablet_service.cc:1467] Processing CreateTablet for tablet a65149e561d9430fab95f8c66beb92f2 (DEFAULT_TABLE table=test-table [id=9947891f439e40ef96e584cf7f6b839e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:39.164278 21738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a65149e561d9430fab95f8c66beb92f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:39.164749 22035 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Bootstrap starting.
I20260811 10:12:39.165225 22035 tablet_bootstrap.cc:654] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:39.165815 22035 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: No bootstrap required, opened a new log
I20260811 10:12:39.165846 22041 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Bootstrap starting.
I20260811 10:12:39.165892 22035 ts_tablet_manager.cc:1397] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:39.166014 22037 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: Bootstrap starting.
I20260811 10:12:39.166052 22035 raft_consensus.cc:348] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.166157 22035 raft_consensus.cc:374] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:39.166186 22035 raft_consensus.cc:729] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf56422974ea423085ae144260bca2e8, State: Initialized, Role: FOLLOWER
I20260811 10:12:39.166249 22041 tablet_bootstrap.cc:654] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:39.166249 22035 consensus_queue.cc:260] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.166436 22037 tablet_bootstrap.cc:654] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:39.166443 22035 ts_tablet_manager.cc:1428] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:39.166917 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.166809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:56629)
0811 10:12:39.166857 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.166883 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 10:12:39.167393 22041 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: No bootstrap required, opened a new log
I20260811 10:12:39.167450 22041 ts_tablet_manager.cc:1397] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:39.167465 22037 tablet_bootstrap.cc:492] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: No bootstrap required, opened a new log
I20260811 10:12:39.167560 22041 raft_consensus.cc:348] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.167508 22037 ts_tablet_manager.cc:1397] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:39.167676 22041 raft_consensus.cc:374] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:39.167719 22041 raft_consensus.cc:729] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4781dbf11449c492a52e214a205408, State: Initialized, Role: FOLLOWER
I20260811 10:12:39.167827 22041 consensus_queue.cc:260] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.167853 22037 raft_consensus.cc:348] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.167919 22037 raft_consensus.cc:374] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:39.167944 22037 raft_consensus.cc:729] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2092d73a6ea24855ab50b2d1d3c2d057, State: Initialized, Role: FOLLOWER
I20260811 10:12:39.168006 22037 consensus_queue.cc:260] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.168043 22041 ts_tablet_manager.cc:1428] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:39.168210 22037 ts_tablet_manager.cc:1428] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:12:39.168368 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.168252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:50679)
0811 10:12:39.168296 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.168336 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 10:12:39.168735 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.168640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:39415)
0811 10:12:39.168677 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.168702 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 10:12:39.168854 21848 heartbeater.cc:503] Master 127.8.238.61:37013 was elected leader, sending a full tablet report...
I20260811 10:12:39.169236 21389 heartbeater.cc:503] Master 127.8.238.61:37013 was elected leader, sending a full tablet report...
I20260811 10:12:39.169368 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.169280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:43063)
0811 10:12:39.169313 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.169337 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 10:12:39.193953 22046 raft_consensus.cc:482] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:39.194078 22046 raft_consensus.cc:504] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.194478 22046 leader_election.cc:288] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815), cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
I20260811 10:12:39.198345 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.194705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:32833 (local address 127.8.238.3:33815)
0811 10:12:39.194790 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.194792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.194801 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:39.194832 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.194834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.194847 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.194906 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:39.195132 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.195917 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.197367 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.197948 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.198015 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.198027 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.198245 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.198285 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:12:39.198342 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.194704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:48775 (local address 127.8.238.2:44321)
0811 10:12:39.194766 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.194773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.195119 (+ 346us) server_negotiation.cc:407] Connection header received
0811 10:12:39.195152 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.195154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.195168 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.195227 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:39.195329 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.196027 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.197366 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.197932 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.198015 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.198027 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.198214 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.198270 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:12:39.198374 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.194642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33815 (local address 127.8.238.1:32833)
0811 10:12:39.194682 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.194694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:39.194749 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:39.194930 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:39.194931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:39.194952 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:39.195045 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.195047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.196001 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.196002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.197272 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.197274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.198039 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.198040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.198132 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.198135 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:39.198189 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 10:12:39.198254 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":8}
I20260811 10:12:39.198375 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.194642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:44321 (local address 127.8.238.1:48775)
0811 10:12:39.195012 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.195025 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:39.195092 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:39.195240 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:39.195241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:39.195242 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:39.195279 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.195280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.196099 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.196099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.197271 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:39.197274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.198044 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.198044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:39.198124 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.198130 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:12:39.198189 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:12:39.198254 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":194,"thread_start_us":134,"threads_started":1}
I20260811 10:12:39.198690 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65149e561d9430fab95f8c66beb92f2" candidate_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4781dbf11449c492a52e214a205408" is_pre_election: true
I20260811 10:12:39.198734 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65149e561d9430fab95f8c66beb92f2" candidate_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf56422974ea423085ae144260bca2e8" is_pre_election: true
I20260811 10:12:39.198824 21769 raft_consensus.cc:2454] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2092d73a6ea24855ab50b2d1d3c2d057 in term 0.
I20260811 10:12:39.198859 21545 raft_consensus.cc:2454] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2092d73a6ea24855ab50b2d1d3c2d057 in term 0.
I20260811 10:12:39.199038 21194 leader_election.cc:302] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4781dbf11449c492a52e214a205408, 2092d73a6ea24855ab50b2d1d3c2d057; no voters:
I20260811 10:12:39.199167 22046 raft_consensus.cc:2781] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:39.199216 22046 raft_consensus.cc:482] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:39.199236 22046 raft_consensus.cc:3037] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:39.200059 22046 raft_consensus.cc:504] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.200207 22046 leader_election.cc:288] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [CANDIDATE]: Term 1 election: Requested vote from peers 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815), cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321)
I20260811 10:12:39.200361 21545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65149e561d9430fab95f8c66beb92f2" candidate_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf56422974ea423085ae144260bca2e8"
I20260811 10:12:39.200387 21769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a65149e561d9430fab95f8c66beb92f2" candidate_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e4781dbf11449c492a52e214a205408"
I20260811 10:12:39.200439 21545 raft_consensus.cc:3037] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:39.200469 21769 raft_consensus.cc:3037] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:39.201247 21769 raft_consensus.cc:2454] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2092d73a6ea24855ab50b2d1d3c2d057 in term 1.
I20260811 10:12:39.201390 21545 raft_consensus.cc:2454] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2092d73a6ea24855ab50b2d1d3c2d057 in term 1.
I20260811 10:12:39.201458 21194 leader_election.cc:302] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e4781dbf11449c492a52e214a205408, 2092d73a6ea24855ab50b2d1d3c2d057; no voters:
I20260811 10:12:39.201592 22046 raft_consensus.cc:2781] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:39.201777 22046 raft_consensus.cc:686] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 LEADER]: Becoming Leader. State: Replica: 2092d73a6ea24855ab50b2d1d3c2d057, State: Running, Role: LEADER
I20260811 10:12:39.201862 22046 consensus_queue.cc:237] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } }
I20260811 10:12:39.202106 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.202003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:45169)
0811 10:12:39.202038 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.202075 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 10:12:39.202625 20761 catalog_manager.cc:5495] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2092d73a6ea24855ab50b2d1d3c2d057" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:39.220422 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.217850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.8.238.1:36539)
0811 10:12:39.217895 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:39.217897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:39.217907 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:39.217995 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:39.217995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:39.218015 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:39.218095 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:39.218227 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.218694 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.219564 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:39.219838 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:39.219916 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:39.220071 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:39.220184 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:39.220276 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:39.220317 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 10:12:39.220339 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:39.221666 21769 raft_consensus.cc:1261] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Refusing update from remote peer 2092d73a6ea24855ab50b2d1d3c2d057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:39.221753 21545 raft_consensus.cc:1261] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Refusing update from remote peer 2092d73a6ea24855ab50b2d1d3c2d057: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:39.222021 22048 consensus_queue.cc:1035] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e4781dbf11449c492a52e214a205408" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:39.222179 22046 consensus_queue.cc:1035] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf56422974ea423085ae144260bca2e8" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:39.222672 21624 heartbeater.cc:503] Master 127.8.238.61:37013 was elected leader, sending a full tablet report...
I20260811 10:12:39.223030 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.222913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:35017)
0811 10:12:39.222964 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.222996 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 10:12:39.223188 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.223018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:52641)
0811 10:12:39.223129 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.223157 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 10:12:39.224449 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.224221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:51751)
0811 10:12:39.224386 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.224418 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 10:12:39.224486 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.224405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.3:53615)
0811 10:12:39.224440 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.224460 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:12:39.225373 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.225287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.1:50243)
0811 10:12:39.225317 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.225346 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260811 10:12:39.225250 22052 mvcc.cc:205] Tried to move back new op lower bound from 7317271180169916416 to 7317271180091650048. Current Snapshot: MvccSnapshot[applied={T|T < 7317271180169916416 or (T in {7317271180169916416})}]
I20260811 10:12:39.225996 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.225903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43017 (local address 127.8.238.2:44021)
0811 10:12:39.225943 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.225967 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:43017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 10:12:39.226861 20759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:39.226965 20759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:39.228416 20759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be7a19682dfc40eb99d4093b863a61e3: Sending DeleteTablet for 3 replicas of tablet a65149e561d9430fab95f8c66beb92f2
I20260811 10:12:39.228690 21738 tablet_service.cc:1514] Processing DeleteTablet for tablet a65149e561d9430fab95f8c66beb92f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:39 UTC) from {username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:48950
I20260811 10:12:39.228751 21271 tablet_service.cc:1514] Processing DeleteTablet for tablet a65149e561d9430fab95f8c66beb92f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:39 UTC) from {username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:38272
I20260811 10:12:39.228910 22066 tablet_replica.cc:330] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: stopping tablet replica
I20260811 10:12:39.228901 21519 tablet_service.cc:1514] Processing DeleteTablet for tablet a65149e561d9430fab95f8c66beb92f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:39 UTC) from {username='oryx', principal='oryx/127.8.238.61@KRBTEST.COM'} at 127.0.0.1:55576
I20260811 10:12:39.228953 22067 tablet_replica.cc:330] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: stopping tablet replica
I20260811 10:12:39.229012 22067 raft_consensus.cc:2229] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:39.229007 22066 raft_consensus.cc:2229] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:39.229081 22066 raft_consensus.cc:2258] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:39.229096 22067 raft_consensus.cc:2258] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:39.229485 22067 ts_tablet_manager.cc:1905] T a65149e561d9430fab95f8c66beb92f2 P 2092d73a6ea24855ab50b2d1d3c2d057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:39.229523 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 21013
I20260811 10:12:39.229542 22066 ts_tablet_manager.cc:1905] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:39.230985 22071 tablet_replica.cc:330] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: stopping tablet replica
I20260811 10:12:39.231070 22071 raft_consensus.cc:2229] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:39.231124 22071 raft_consensus.cc:2258] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:39.231237 22066 ts_tablet_manager.cc:1918] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:39.231309 22066 log.cc:1196] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/a65149e561d9430fab95f8c66beb92f2
I20260811 10:12:39.231545 22066 ts_tablet_manager.cc:1939] T a65149e561d9430fab95f8c66beb92f2 P 1e4781dbf11449c492a52e214a205408: Deleting consensus metadata
I20260811 10:12:39.232966 20740 catalog_manager.cc:4841] TS 1e4781dbf11449c492a52e214a205408 (127.8.238.3:33815): tablet a65149e561d9430fab95f8c66beb92f2 (table test-table [id=9947891f439e40ef96e584cf7f6b839e]) successfully deleted
I20260811 10:12:39.234565 22071 ts_tablet_manager.cc:1905] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:39.235957 22071 ts_tablet_manager.cc:1918] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:39.236054 22071 log.cc:1196] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/a65149e561d9430fab95f8c66beb92f2
I20260811 10:12:39.236297 22071 ts_tablet_manager.cc:1939] T a65149e561d9430fab95f8c66beb92f2 P cf56422974ea423085ae144260bca2e8: Deleting consensus metadata
I20260811 10:12:39.236618 20737 catalog_manager.cc:4841] TS cf56422974ea423085ae144260bca2e8 (127.8.238.2:44321): tablet a65149e561d9430fab95f8c66beb92f2 (table test-table [id=9947891f439e40ef96e584cf7f6b839e]) successfully deleted
I20260811 10:12:39.244911 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:39.244813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36539 (local address 127.0.0.1:38286)
0811 10:12:39.244852 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:39.244880 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:36539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260811 10:12:39.245090 20739 catalog_manager.cc:4584] TS 2092d73a6ea24855ab50b2d1d3c2d057 (127.8.238.1:36539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a65149e561d9430fab95f8c66beb92f2: Network error: Client connection negotiation failed: client connection to 127.8.238.1:36539: connect: Connection refused (error 111)
I20260811 10:12:39.245183 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 21399
I20260811 10:12:39.254190 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 21635
I20260811 10:12:39.264606 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 20549
I20260811 10:12:39.275141 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 20853
2026-08-11T10:12:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6604 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:39 dist-test-slave-7f01 krb5kdc[22075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:39 dist-test-slave-7f01 krb5kdc[22075](info): set up 2 sockets
Aug 11 10:12:39 dist-test-slave-7f01 krb5kdc[22075](info): commencing operation
krb5kdc: starting...
W20260811 10:12:40.565049 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:40 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab.
Entry for principal oryx/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab.
I20260811 10:12:40.691246 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45573
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab
--principal=oryx/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:45573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:40.775744 22100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:40.775985 22100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:40.776010 22100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:40.777222 22100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:40.777266 22100 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 10:12:40.777280 22100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:40.777293 22100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:40.777331 22100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:40.777364 22100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:40.778900 22100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:45573
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.62.keytab
--principal=oryx/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:40.779167 22100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:40.779412 22100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:40.784497 22111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:40.784569 22100 server_base.cc:1034] running on GCE node
W20260811 10:12:40.784513 22110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:40.784507 22113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:40.785023 22100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:40.785336 22100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:40.786476 22100 hybrid_clock.cc:648] HybridClock initialized: now 1786443160786452 us; error 38 us; skew 500 ppm
Aug 11 10:12:40 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443160, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:40.789150 22100 init.cc:377] Logged in from keytab as oryx/127.8.238.62@KRBTEST.COM (short username oryx)
I20260811 10:12:40.789636 22100 webserver.cc:460] Webserver started at http://127.8.238.62:40899/ using document root <none> and password file <none>
I20260811 10:12:40.789892 22100 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:40.789939 22100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:40.790082 22100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:40.791136 22100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ff13f86463124373aa071b3862a646e8"
format_stamp: "Formatted at 2026-08-11 10:12:40 on dist-test-slave-7f01"
server_key: "326f0d0a13ca687942110dc85420f700"
server_key_iv: "a6cb2c5a3db756a2c4123417b0caa791"
server_key_version: "encryptionkey@0"
I20260811 10:12:40.791527 22100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ff13f86463124373aa071b3862a646e8"
format_stamp: "Formatted at 2026-08-11 10:12:40 on dist-test-slave-7f01"
server_key: "326f0d0a13ca687942110dc85420f700"
server_key_iv: "a6cb2c5a3db756a2c4123417b0caa791"
server_key_version: "encryptionkey@0"
I20260811 10:12:40.792963 22100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:40.793733 22125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:40.793994 22100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:40.794123 22100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ff13f86463124373aa071b3862a646e8"
format_stamp: "Formatted at 2026-08-11 10:12:40 on dist-test-slave-7f01"
server_key: "326f0d0a13ca687942110dc85420f700"
server_key_iv: "a6cb2c5a3db756a2c4123417b0caa791"
server_key_version: "encryptionkey@0"
I20260811 10:12:40.794209 22100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:40.826552 22100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:40.828672 22100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:40.828857 22100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:40.834435 22100 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:45573
I20260811 10:12:40.834494 22226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:45573 every 8 connection(s)
I20260811 10:12:40.834877 22100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:40.835546 22227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:40.836647 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 22100
I20260811 10:12:40.836794 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:40.837023 9144 external_mini_cluster.cc:1428] Setting key 1845272039e04253683b27e27e0add2a
I20260811 10:12:40.838557 22227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Bootstrap starting.
I20260811 10:12:40.839216 22227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:40.839607 22227 log.cc:824] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:40.840435 22227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: No bootstrap required, opened a new log
I20260811 10:12:40.842833 22227 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } }
I20260811 10:12:40.843045 22227 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:40.843112 22227 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff13f86463124373aa071b3862a646e8, State: Initialized, Role: FOLLOWER
I20260811 10:12:40.843245 22227 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } }
I20260811 10:12:40.843349 22227 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:40.843398 22227 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:40.843451 22227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:40.844081 22227 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } }
I20260811 10:12:40.844203 22227 leader_election.cc:302] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff13f86463124373aa071b3862a646e8; no voters:
I20260811 10:12:40.844377 22227 leader_election.cc:288] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:40.844444 22235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:40.844610 22235 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [term 1 LEADER]: Becoming Leader. State: Replica: ff13f86463124373aa071b3862a646e8, State: Running, Role: LEADER
I20260811 10:12:40.844677 22227 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 10:12:40 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:40.844753 22235 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } }
I20260811 10:12:40.845106 22238 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff13f86463124373aa071b3862a646e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } } }
I20260811 10:12:40.845113 22236 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff13f86463124373aa071b3862a646e8. Latest consensus state: current_term: 1 leader_uuid: "ff13f86463124373aa071b3862a646e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff13f86463124373aa071b3862a646e8" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45573 } } }
I20260811 10:12:40.845201 22238 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:40.845206 22236 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8 [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:40.845407 22243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:40.846024 22243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:40.847615 22243 catalog_manager.cc:1347] Generated new cluster ID: 3c426bdbce124ef3a9f69bc0b3506a02
I20260811 10:12:40.847707 22243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:40.849336 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:40.837639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56872 (local address 127.8.238.62:45573)
0811 10:12:40.837849 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:40.837858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:40.837894 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:12:40.838446 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:40.838456 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:40.838529 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:40.838685 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:40.839486 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:40.840887 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:40.841679 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:40.842323 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:40.842442 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:40.845545 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:40.845550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:40.845550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:40.845563 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:40.847694 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:40.848106 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:40.848109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:40.848110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:40.848206 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:40.848431 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:40.848432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:40.848432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:40.848673 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:40.848852 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:40.848942 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 10:12:40.849182 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260811 10:12:40.855068 22243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:40.855641 22243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:40.864455 22243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Generated new TSK 0
I20260811 10:12:40.864645 22243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
I20260811 10:12:40.904597 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45573
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:41.000825 22267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:41.001044 22267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:41.001070 22267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:41.002235 22267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:41.002276 22267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:41.002321 22267 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:41.002358 22267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:41.003857 22267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:45573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:41.004109 22267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:41.004364 22267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:12:41.009449 22267 server_base.cc:1034] running on GCE node
W20260811 10:12:41.009393 22278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.009385 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.009441 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:41.009850 22267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:41.010097 22267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:41.011260 22267 hybrid_clock.cc:648] HybridClock initialized: now 1786443161011259 us; error 53 us; skew 500 ppm
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:41.013917 22267 init.cc:377] Logged in from keytab as oryx/127.8.238.1@KRBTEST.COM (short username oryx)
I20260811 10:12:41.014343 22267 webserver.cc:460] Webserver started at http://127.8.238.1:37815/ using document root <none> and password file <none>
I20260811 10:12:41.014585 22267 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:41.014655 22267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:41.014816 22267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:41.015921 22267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "c5f08c3946e9414d80db070344ab0f18"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "a9a2cf482f2abeff9b840ce6240af69a"
server_key_iv: "b8255a108fa45a0705e1a1b407ccab41"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.016306 22267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "c5f08c3946e9414d80db070344ab0f18"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "a9a2cf482f2abeff9b840ce6240af69a"
server_key_iv: "b8255a108fa45a0705e1a1b407ccab41"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.017637 22267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:41.018356 22294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.018532 22267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:41.018641 22267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "c5f08c3946e9414d80db070344ab0f18"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "a9a2cf482f2abeff9b840ce6240af69a"
server_key_iv: "b8255a108fa45a0705e1a1b407ccab41"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.018715 22267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:41.034512 22267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:41.036569 22267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:41.036733 22267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:41.037235 22267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:41.037307 22267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.037350 22267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:41.037400 22267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:41.045418 22267 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34015
I20260811 10:12:41.045536 22484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34015 every 8 connection(s)
I20260811 10:12:41.045898 22267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:41.048053 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.037814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39833 (local address 127.8.238.62:45573)
0811 10:12:41.037891 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.037893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.038101 (+ 208us) server_negotiation.cc:407] Connection header received
0811 10:12:41.038602 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.038603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.038617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.038698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.039639 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.040531 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.041948 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.042224 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.042307 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.044992 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.044993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.044994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.045000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.046828 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.047207 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.047208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.047209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.047265 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.047483 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.047484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.047484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.047583 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.047629 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.047905 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.047986 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 10:12:41.048204 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.037779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.1:39833)
0811 10:12:41.037961 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.038002 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.038496 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.038809 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.038813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.039178 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.039516 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.039523 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.040684 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.040685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.041841 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.041844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.042329 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.042332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.042467 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.042998 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.043015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.044893 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.046899 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.046904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.046906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.047138 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.047329 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.047330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.047330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.047429 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.047634 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.047638 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.047791 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.047890 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.048024 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260811 10:12:41.048745 22486 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45573
I20260811 10:12:41.048832 22486 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:41.049015 22486 heartbeater.cc:511] Master 127.8.238.62:45573 requested a full tablet report, sending...
I20260811 10:12:41.049677 22154 ts_manager.cc:194] Registered new tserver with Master: c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015)
I20260811 10:12:41.050346 22154 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:39833
I20260811 10:12:41.050881 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 22267
I20260811 10:12:41.051031 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:41.051292 9144 external_mini_cluster.cc:1428] Setting key 8388e562050094d5b1ae26cc0e20dcb0
I20260811 10:12:41.056921 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.049811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:36151 (local address 127.8.238.62:45573)
0811 10:12:41.049849 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.049851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.050674 (+ 823us) server_negotiation.cc:407] Connection header received
0811 10:12:41.050803 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.050805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.050818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.050853 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.051324 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.052046 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.053013 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.053193 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.053244 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.054269 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.054271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.054271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.054278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.055821 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.056221 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.056221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.056222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.056272 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.056477 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.056477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.056477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.056570 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.056618 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.056799 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.056862 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":20}
I20260811 10:12:41.057049 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.049724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.1:36151)
0811 10:12:41.050619 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.050629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.050717 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.050922 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.050923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.051168 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.051244 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.051246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.052127 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.052128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.052931 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.052935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.053266 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.053267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.053348 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.053721 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.053726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.054176 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.055898 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.055899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.055900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.056146 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.056337 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.056338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.056339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.056415 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.056632 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.056633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.056700 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.056774 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.056889 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
I20260811 10:12:41.082214 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45573
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:41.167788 22507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:41.168002 22507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:41.168026 22507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:41.169139 22507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:41.169173 22507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:41.169193 22507 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:41.169211 22507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:41.170640 22507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:45573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:41.170820 22507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:41.170992 22507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:41.176054 22519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.176069 22514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.176054 22516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:41.176394 22507 server_base.cc:1034] running on GCE node
I20260811 10:12:41.176566 22507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:41.176751 22507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:41.177889 22507 hybrid_clock.cc:648] HybridClock initialized: now 1786443161177881 us; error 45 us; skew 500 ppm
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:41.180516 22507 init.cc:377] Logged in from keytab as oryx/127.8.238.2@KRBTEST.COM (short username oryx)
I20260811 10:12:41.181013 22507 webserver.cc:460] Webserver started at http://127.8.238.2:45155/ using document root <none> and password file <none>
I20260811 10:12:41.181255 22507 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:41.181301 22507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:41.181392 22507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:41.182364 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "e53d7e22ccf04435b2987f319db85123"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "f8b75199b6bb1bcbad94369911e6309e"
server_key_iv: "040a18494d43f93bf11e998f8ec34b59"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.182679 22507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "e53d7e22ccf04435b2987f319db85123"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "f8b75199b6bb1bcbad94369911e6309e"
server_key_iv: "040a18494d43f93bf11e998f8ec34b59"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.184034 22507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:41.184783 22527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.184975 22507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.185074 22507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "e53d7e22ccf04435b2987f319db85123"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "f8b75199b6bb1bcbad94369911e6309e"
server_key_iv: "040a18494d43f93bf11e998f8ec34b59"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.185150 22507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:41.202983 22507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:41.205505 22507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:41.205675 22507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:41.206350 22507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:41.206410 22507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.206444 22507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:41.206465 22507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:41.214425 22507 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:38111
I20260811 10:12:41.214504 22709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:38111 every 8 connection(s)
I20260811 10:12:41.214859 22507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:12:41.216692 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.206811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:54883 (local address 127.8.238.62:45573)
0811 10:12:41.206884 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.206887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.207137 (+ 250us) server_negotiation.cc:407] Connection header received
0811 10:12:41.207650 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.207652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.207667 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.207778 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.208379 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.209258 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.210286 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.210522 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.210612 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.213463 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.213465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.213465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.213473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.215224 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.215670 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.215671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.215672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.215762 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.216017 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.216018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.216018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.216148 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.216227 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.216506 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.216598 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:12:41.216874 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.206785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.2:54883)
0811 10:12:41.206997 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.207038 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.207549 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.207855 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.207858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.208098 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.208288 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.208293 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.209376 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.209377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.210188 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.210190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.210625 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.210626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.210758 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.211223 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.211231 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.213364 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.215315 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.215320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.215322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.215582 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.215817 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.215818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.215819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.215934 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.216217 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.216225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.216366 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.216486 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.216648 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":136,"threads_started":1}
I20260811 10:12:41.217382 22711 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45573
I20260811 10:12:41.217494 22711 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:41.217646 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 22507
I20260811 10:12:41.217676 22711 heartbeater.cc:511] Master 127.8.238.62:45573 requested a full tablet report, sending...
I20260811 10:12:41.217772 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:41.218017 9144 external_mini_cluster.cc:1428] Setting key d29d7bb39c9131e187be1cb33bcc1ab4
I20260811 10:12:41.218277 22155 ts_manager.cc:194] Registered new tserver with Master: e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:41.218837 22155 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:54883
I20260811 10:12:41.225421 22720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.218498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.2:41391)
0811 10:12:41.219084 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.219096 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.219154 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.219357 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.219358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.219598 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.219690 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.219692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.220359 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.220360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.221083 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.221086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.221583 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.221584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.221645 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.222104 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.222108 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.222434 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.224335 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.224336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.224336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.224494 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.224711 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.224712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.224712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.224797 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.225171 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.225172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.225208 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.225267 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.225353 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":72,"threads_started":1}
I20260811 10:12:41.225658 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.218550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:41391 (local address 127.8.238.62:45573)
0811 10:12:41.218587 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.218588 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.219155 (+ 567us) server_negotiation.cc:407] Connection header received
0811 10:12:41.219257 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.219258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.219270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.219299 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.219800 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.220260 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.221162 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.221503 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.221574 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.222517 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.222518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.222519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.222526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.224246 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.224572 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.224574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.224574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.224642 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.224924 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.224926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.224926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.225089 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.225128 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.225333 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.225426 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for oryx/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
I20260811 10:12:41.246505 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45573
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:41.329916 22726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:41.330118 22726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:41.330142 22726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:41.331239 22726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:41.331271 22726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:41.331291 22726 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 10:12:41.331308 22726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:41.332756 22726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.238.62:45573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:41.332935 22726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:41.333105 22726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:41.338137 22739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.338186 22737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:41.338137 22736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:41.338363 22726 server_base.cc:1034] running on GCE node
I20260811 10:12:41.338599 22726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:41.338837 22726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:41.340008 22726 hybrid_clock.cc:648] HybridClock initialized: now 1786443161340005 us; error 39 us; skew 500 ppm
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:41.342610 22726 init.cc:377] Logged in from keytab as oryx/127.8.238.3@KRBTEST.COM (short username oryx)
I20260811 10:12:41.343061 22726 webserver.cc:460] Webserver started at http://127.8.238.3:39761/ using document root <none> and password file <none>
I20260811 10:12:41.343299 22726 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:41.343369 22726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:41.343523 22726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:41.344754 22726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "ca4cedda4ed94a6287e0f561b6611f47"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "8268a19b148bb2f341fa0df5cd4b50e5"
server_key_iv: "f6ea29a5d387bc6e1ea3c9c7138b70ec"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.345160 22726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "ca4cedda4ed94a6287e0f561b6611f47"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "8268a19b148bb2f341fa0df5cd4b50e5"
server_key_iv: "f6ea29a5d387bc6e1ea3c9c7138b70ec"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.346697 22726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:41.347409 22749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.347612 22726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.347692 22726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "ca4cedda4ed94a6287e0f561b6611f47"
format_stamp: "Formatted at 2026-08-11 10:12:41 on dist-test-slave-7f01"
server_key: "8268a19b148bb2f341fa0df5cd4b50e5"
server_key_iv: "f6ea29a5d387bc6e1ea3c9c7138b70ec"
server_key_version: "encryptionkey@0"
I20260811 10:12:41.347808 22726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:41.359242 22726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:41.361135 22726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:41.361295 22726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:41.361829 22726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:41.361901 22726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.361948 22726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:41.361975 22726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.369122 22726 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:36739
I20260811 10:12:41.369215 22939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:36739 every 8 connection(s)
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:41.369547 22726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:41.371543 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 22726
I20260811 10:12:41.371661 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:41.371882 9144 external_mini_cluster.cc:1428] Setting key a8428bb13ea198d96bd027dfe7617acf
I20260811 10:12:41.372814 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.362314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:56763 (local address 127.8.238.62:45573)
0811 10:12:41.362386 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.362389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.362583 (+ 194us) server_negotiation.cc:407] Connection header received
0811 10:12:41.363065 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.363067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.363080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.363168 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.364080 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.365031 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.366612 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.366991 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.367083 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.369997 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.369998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.369998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.370006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.371185 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.371598 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.371600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.371600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.371674 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.372034 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.372035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.372035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.372195 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.372277 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.372613 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.372713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 10:12:41.372908 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.362288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.3:56763)
0811 10:12:41.362438 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.362485 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.362964 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.363216 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.363219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.363593 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.363983 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.363991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.365129 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.365130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.366500 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.366504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.367118 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.367119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.367264 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.367865 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.367876 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.369904 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.371260 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.371269 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.371271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.371494 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.371759 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.371760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.371760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.371930 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.372273 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.372281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.372447 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.372559 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.372718 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260811 10:12:41.373371 22940 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45573
I20260811 10:12:41.373473 22940 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:41.373647 22940 heartbeater.cc:511] Master 127.8.238.62:45573 requested a full tablet report, sending...
I20260811 10:12:41.374090 22154 ts_manager.cc:194] Registered new tserver with Master: ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739)
I20260811 10:12:41.374513 22154 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:56763
I20260811 10:12:41.379436 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.374576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:55269 (local address 127.8.238.62:45573)
0811 10:12:41.374613 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.374615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.374744 (+ 129us) server_negotiation.cc:407] Connection header received
0811 10:12:41.374791 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.374792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.374803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.374864 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.375225 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.375667 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.376365 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.376547 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.376602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.377418 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.377419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.377419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.377425 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.378556 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.378855 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.378855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.378856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.378906 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.379078 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.379078 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.379078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.379169 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.379216 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.379323 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.379377 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 10:12:41.379444 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.374486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.8.238.3:55269)
0811 10:12:41.374672 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.374682 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.374736 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.374868 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.374869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.375085 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.375151 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.375153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.375791 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.375792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.376292 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.376294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.376617 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.376618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.376665 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.377066 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.377070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.377331 (+ 261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.378626 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.378627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.378627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.378791 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.378964 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.378965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.378965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.379025 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.379221 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.379222 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.379256 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.379305 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.379377 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260811 10:12:41.383458 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:12:41.406811 22233 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:41.401216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56876 (local address 127.8.238.62:45573)
0811 10:12:41.401286 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.401288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.401298 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:41.401326 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.401327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.401345 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.401411 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.402010 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.402538 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.403295 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.403494 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.403582 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.406700 (+ 3118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56876: BlockingRecv error: recv got EOF from 127.0.0.1:56876 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:41.413847 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.407511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.8.238.62:45573)
0811 10:12:41.407565 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.407567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.407616 (+ 49us) server_negotiation.cc:407] Connection header received
0811 10:12:41.407688 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.407690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.407699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.407765 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.408233 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.408693 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.409356 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.409557 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.409630 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.411547 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.411548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.411548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.411555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.412745 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.413098 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.413099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.413100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.413157 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.413365 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.413366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.413366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.413473 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.413580 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.413675 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.413747 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:41.415447 22154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:41.417176 22154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:41.424019 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.420059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.8.238.1:34015)
0811 10:12:41.420332 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.420336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.420347 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:41.420389 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.420391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.420410 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.420634 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.420696 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421697 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423070 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.423572 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423660 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.423740 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.423881 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.423922 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":110,"threads_started":1}
I20260811 10:12:41.424006 22982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.419669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34015 (local address 127.0.0.1:57688)
0811 10:12:41.419860 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.419899 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.419961 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.420512 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.420513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.420515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.420560 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.420561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421821 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.421822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.422985 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.422987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.423661 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.423771 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.423789 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.423864 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.423919 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":49,"thread_start_us":130,"threads_started":1}
I20260811 10:12:41.424207 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.419730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.8.238.3:36739)
0811 10:12:41.419985 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.419991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.420004 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:12:41.420052 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.420054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.420071 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.420152 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.420355 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421479 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.422862 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.423571 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423640 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.423714 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.424053 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.424098 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":97,"threads_started":1}
I20260811 10:12:41.424253 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.419882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:38111 (local address 127.0.0.1:47572)
0811 10:12:41.420394 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.420405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.420450 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.420748 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.420749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.420751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.420798 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.420799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421803 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.421805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.422977 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.422980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.424026 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.424027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.424088 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.424091 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.424140 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.424181 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":51,"thread_start_us":82,"threads_started":1}
I20260811 10:12:41.424423 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.419610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.0.0.1:45554)
0811 10:12:41.419854 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.419897 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.419961 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.420175 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.420178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.420180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.420243 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.420245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421572 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.421573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.422776 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.422779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.423798 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.423865 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.423870 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.423919 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.423961 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260811 10:12:41.424541 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.420203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.8.238.2:38111)
0811 10:12:41.420585 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.420589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.420598 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:41.420642 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.420643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.420655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.420739 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.420893 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.421734 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423080 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.423853 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.423922 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.423991 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.424362 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.424411 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":82,"threads_started":1}
I20260811 10:12:41.425033 22838 tablet_service.cc:1467] Processing CreateTablet for tablet 861315d8ee914231b7d47b0f4044ce25 (DEFAULT_TABLE table=test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.425243 22371 tablet_service.cc:1467] Processing CreateTablet for tablet 861315d8ee914231b7d47b0f4044ce25 (DEFAULT_TABLE table=test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.425395 22838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 861315d8ee914231b7d47b0f4044ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.425588 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 861315d8ee914231b7d47b0f4044ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.428478 22996 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Bootstrap starting.
I20260811 10:12:41.428906 22999 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Bootstrap starting.
I20260811 10:12:41.429245 22996 tablet_bootstrap.cc:654] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.429488 22999 tablet_bootstrap.cc:654] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.429392 22608 tablet_service.cc:1467] Processing CreateTablet for tablet 861315d8ee914231b7d47b0f4044ce25 (DEFAULT_TABLE table=test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.429611 22996 log.cc:824] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:41.429770 22608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 861315d8ee914231b7d47b0f4044ce25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.429858 22999 log.cc:824] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:41.430538 22996 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: No bootstrap required, opened a new log
I20260811 10:12:41.430619 22996 ts_tablet_manager.cc:1397] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:41.430749 22999 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: No bootstrap required, opened a new log
I20260811 10:12:41.430858 22999 ts_tablet_manager.cc:1397] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:41.432721 22996 raft_consensus.cc:348] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.432914 22996 raft_consensus.cc:374] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.432962 22996 raft_consensus.cc:729] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.432997 23002 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Bootstrap starting.
I20260811 10:12:41.433075 22996 consensus_queue.cc:260] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.433146 22999 raft_consensus.cc:348] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.433372 22996 ts_tablet_manager.cc:1428] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:41.433409 22940 heartbeater.cc:503] Master 127.8.238.62:45573 was elected leader, sending a full tablet report...
I20260811 10:12:41.433495 22999 raft_consensus.cc:374] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.433553 22999 raft_consensus.cc:729] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.433645 23002 tablet_bootstrap.cc:654] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.433674 22999 consensus_queue.cc:260] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.433950 22999 ts_tablet_manager.cc:1428] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:12:41.434039 23002 log.cc:824] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:41.434176 22486 heartbeater.cc:503] Master 127.8.238.62:45573 was elected leader, sending a full tablet report...
I20260811 10:12:41.435035 23002 tablet_bootstrap.cc:492] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: No bootstrap required, opened a new log
I20260811 10:12:41.435091 23002 ts_tablet_manager.cc:1397] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:41.436616 23002 raft_consensus.cc:348] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.436738 23002 raft_consensus.cc:374] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.436758 23002 raft_consensus.cc:729] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.436894 23002 consensus_queue.cc:260] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.437135 23002 ts_tablet_manager.cc:1428] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:12:41.437189 22711 heartbeater.cc:503] Master 127.8.238.62:45573 was elected leader, sending a full tablet report...
W20260811 10:12:41.465502 22712 tablet.cc:2369] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:41.540287 23004 raft_consensus.cc:482] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:41.540503 23004 raft_consensus.cc:504] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.540894 23004 leader_election.cc:288] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:41.543843 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.541325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:54915 (local address 127.8.238.3:36739)
0811 10:12:41.541374 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.541376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.541383 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:12:41.541420 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.541422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.541431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.541500 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.541671 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.542152 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543090 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.543447 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543547 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.543560 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.543705 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.543773 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:41.543871 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.541085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.8.238.1:54915)
0811 10:12:41.541252 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.541266 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.541326 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.541519 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.541520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.541522 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.541598 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.541600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.542246 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.542247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.542995 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.542997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.543548 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.543629 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.543632 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.543699 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.543743 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 10:12:41.543979 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.541311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:49049 (local address 127.8.238.2:38111)
0811 10:12:41.541361 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.541363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.541408 (+ 45us) server_negotiation.cc:407] Connection header received
0811 10:12:41.541497 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.541498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.541508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.541565 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.541742 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.542205 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543117 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.543542 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543624 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.543635 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.543835 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.543872 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:12:41.544206 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.541199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:38111 (local address 127.8.238.1:49049)
0811 10:12:41.541330 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.541341 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.541415 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.541582 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.541583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.541585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.541666 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.541668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.542298 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.542299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.543020 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.543022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.543629 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.543630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.543710 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.543713 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.543825 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.543865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260811 10:12:41.544414 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861315d8ee914231b7d47b0f4044ce25" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47" is_pre_election: true
I20260811 10:12:41.544631 22872 raft_consensus.cc:2454] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 0.
I20260811 10:12:41.544569 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861315d8ee914231b7d47b0f4044ce25" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123" is_pre_election: true
I20260811 10:12:41.544760 22644 raft_consensus.cc:2454] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 0.
I20260811 10:12:41.544915 22298 leader_election.cc:302] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, ca4cedda4ed94a6287e0f561b6611f47; no voters:
I20260811 10:12:41.545089 23004 raft_consensus.cc:2781] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:41.545169 23004 raft_consensus.cc:482] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:41.545187 23004 raft_consensus.cc:3037] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.546135 23004 raft_consensus.cc:504] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.546267 23004 leader_election.cc:288] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 election: Requested vote from peers ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:41.546468 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861315d8ee914231b7d47b0f4044ce25" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47"
I20260811 10:12:41.546510 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "861315d8ee914231b7d47b0f4044ce25" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123"
I20260811 10:12:41.546572 22644 raft_consensus.cc:3037] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.546568 22872 raft_consensus.cc:3037] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.547323 22872 raft_consensus.cc:2454] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 1.
I20260811 10:12:41.547343 22644 raft_consensus.cc:2454] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 1.
W20260811 10:12:41.547423 22488 tablet.cc:2369] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:41.547526 22298 leader_election.cc:302] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:41.547617 23004 raft_consensus.cc:2781] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:41.547825 23004 raft_consensus.cc:686] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 1 LEADER]: Becoming Leader. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Running, Role: LEADER
I20260811 10:12:41.547926 23004 consensus_queue.cc:237] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } }
I20260811 10:12:41.548825 22155 catalog_manager.cc:5495] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5f08c3946e9414d80db070344ab0f18 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "c5f08c3946e9414d80db070344ab0f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } health_report { overall_health: HEALTHY } } }
I20260811 10:12:41.567376 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.565287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.8.238.1:34015)
0811 10:12:41.565348 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.565350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.565357 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:12:41.565388 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.565389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.565407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.565469 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:41.565592 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.566043 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.566744 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.566931 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.566983 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.567117 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:41.567212 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:41.567269 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.567303 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.567320 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:12:41.569126 22644 raft_consensus.cc:1261] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Refusing update from remote peer c5f08c3946e9414d80db070344ab0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:41.569140 22872 raft_consensus.cc:1261] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Refusing update from remote peer c5f08c3946e9414d80db070344ab0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:41.569470 23004 consensus_queue.cc:1035] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:41.569625 23008 consensus_queue.cc:1035] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:41.571923 23011 mvcc.cc:205] Tried to move back new op lower bound from 7317271189784412160 to 7317271189701021696. Current Snapshot: MvccSnapshot[applied={T|T < 7317271189784412160}]
I20260811 10:12:41.572237 23012 mvcc.cc:205] Tried to move back new op lower bound from 7317271189784412160 to 7317271189701021696. Current Snapshot: MvccSnapshot[applied={T|T < 7317271189784412160}]
I20260811 10:12:41.574784 22155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:41.574898 22155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:41.575696 22155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Sending DeleteTablet for 3 replicas of tablet 861315d8ee914231b7d47b0f4044ce25
I20260811 10:12:41.576115 22371 tablet_service.cc:1514] Processing DeleteTablet for tablet 861315d8ee914231b7d47b0f4044ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:41 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57688
I20260811 10:12:41.576180 22838 tablet_service.cc:1514] Processing DeleteTablet for tablet 861315d8ee914231b7d47b0f4044ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:41 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 10:12:41.576330 22608 tablet_service.cc:1514] Processing DeleteTablet for tablet 861315d8ee914231b7d47b0f4044ce25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:41 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47572
I20260811 10:12:41.576447 23027 tablet_replica.cc:330] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: stopping tablet replica
I20260811 10:12:41.576539 23027 raft_consensus.cc:2229] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:41.576630 23029 tablet_replica.cc:330] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: stopping tablet replica
I20260811 10:12:41.576623 22155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 10:12:41.576719 23027 raft_consensus.cc:2258] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 10:12:41.576773 22155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:41.576776 23029 raft_consensus.cc:2229] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:41.576881 23029 raft_consensus.cc:2258] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:41.577193 23027 ts_tablet_manager.cc:1905] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:41.577322 23029 ts_tablet_manager.cc:1905] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:41.578330 22371 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d (DEFAULT_TABLE table=test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.578449 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3319ae8e5743149552124d0f2f9e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.578960 23027 ts_tablet_manager.cc:1918] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:41.579041 23027 log.cc:1196] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/861315d8ee914231b7d47b0f4044ce25
I20260811 10:12:41.579077 23029 ts_tablet_manager.cc:1918] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:41.579146 23029 log.cc:1196] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/861315d8ee914231b7d47b0f4044ce25
I20260811 10:12:41.579324 23027 ts_tablet_manager.cc:1939] T 861315d8ee914231b7d47b0f4044ce25 P c5f08c3946e9414d80db070344ab0f18: Deleting consensus metadata
I20260811 10:12:41.579416 23029 ts_tablet_manager.cc:1939] T 861315d8ee914231b7d47b0f4044ce25 P e53d7e22ccf04435b2987f319db85123: Deleting consensus metadata
I20260811 10:12:41.579710 22133 catalog_manager.cc:4841] TS c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015): tablet 861315d8ee914231b7d47b0f4044ce25 (table test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]) successfully deleted
I20260811 10:12:41.579849 22999 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Bootstrap starting.
I20260811 10:12:41.579977 22131 catalog_manager.cc:4841] TS e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111): tablet 861315d8ee914231b7d47b0f4044ce25 (table test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]) successfully deleted
I20260811 10:12:41.580080 22838 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d (DEFAULT_TABLE table=test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.580193 22838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3319ae8e5743149552124d0f2f9e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.580300 22999 tablet_bootstrap.cc:654] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.580830 22608 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d (DEFAULT_TABLE table=test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:41.580936 22608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3319ae8e5743149552124d0f2f9e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.581035 23028 tablet_replica.cc:330] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: stopping tablet replica
I20260811 10:12:41.581125 23028 raft_consensus.cc:2229] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:41.581221 23028 raft_consensus.cc:2258] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:41.581604 22996 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Bootstrap starting.
I20260811 10:12:41.582063 22996 tablet_bootstrap.cc:654] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.582208 23002 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Bootstrap starting.
I20260811 10:12:41.582278 22999 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: No bootstrap required, opened a new log
I20260811 10:12:41.582319 22999 ts_tablet_manager.cc:1397] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:41.582479 22999 raft_consensus.cc:348] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.582572 22999 raft_consensus.cc:374] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.582593 22999 raft_consensus.cc:729] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.582638 22999 consensus_queue.cc:260] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.582731 23002 tablet_bootstrap.cc:654] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.582760 22999 ts_tablet_manager.cc:1428] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.582759 22996 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: No bootstrap required, opened a new log
I20260811 10:12:41.582806 22996 ts_tablet_manager.cc:1397] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.582984 22996 raft_consensus.cc:348] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.583065 22996 raft_consensus.cc:374] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.583086 22996 raft_consensus.cc:729] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.583133 22996 consensus_queue.cc:260] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.583232 22996 ts_tablet_manager.cc:1428] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.583328 23002 tablet_bootstrap.cc:492] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: No bootstrap required, opened a new log
I20260811 10:12:41.583382 23002 ts_tablet_manager.cc:1397] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.583415 23028 ts_tablet_manager.cc:1905] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:41.583541 23002 raft_consensus.cc:348] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.583623 23002 raft_consensus.cc:374] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.583645 23002 raft_consensus.cc:729] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.583693 23002 consensus_queue.cc:260] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.583815 23002 ts_tablet_manager.cc:1428] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:41.584949 23028 ts_tablet_manager.cc:1918] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:41.585040 23028 log.cc:1196] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/861315d8ee914231b7d47b0f4044ce25
I20260811 10:12:41.585274 23028 ts_tablet_manager.cc:1939] T 861315d8ee914231b7d47b0f4044ce25 P ca4cedda4ed94a6287e0f561b6611f47: Deleting consensus metadata
I20260811 10:12:41.585655 22131 catalog_manager.cc:4841] TS ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739): tablet 861315d8ee914231b7d47b0f4044ce25 (table test-table [id=7fbd49cbdc484b2a827006b00e8b7d56]) successfully deleted
W20260811 10:12:41.620138 22942 tablet.cc:2369] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:12:41.851962 23003 raft_consensus.cc:482] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:41.852074 23003 raft_consensus.cc:504] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.852476 23003 leader_election.cc:288] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:41.855531 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.852675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:38111 (local address 127.8.238.3:38203)
0811 10:12:41.852810 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.852823 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.852886 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.853049 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.853050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.853051 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.853139 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.853142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.853833 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.853835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.854697 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.854700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.855250 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.855251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.855329 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.855334 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.855384 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.855418 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 10:12:41.855561 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.852746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:38203 (local address 127.8.238.2:38111)
0811 10:12:41.852824 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.852826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.852885 (+ 59us) server_negotiation.cc:407] Connection header received
0811 10:12:41.852964 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.852965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.852975 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.853034 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.853227 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.853747 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.854768 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.855169 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.855225 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.855235 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.855416 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.855472 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:12:41.855605 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.852835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:41787 (local address 127.8.238.1:34015)
0811 10:12:41.852884 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.852886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.852976 (+ 90us) server_negotiation.cc:407] Connection header received
0811 10:12:41.853052 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.853053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.853064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.853119 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:41.853306 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.853761 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.854827 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.855188 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.855280 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.855291 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.855478 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.855524 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:41.855652 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.852746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34015 (local address 127.8.238.3:41787)
0811 10:12:41.852888 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.852899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.852967 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.853128 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.853129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.853130 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:41.853232 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.853233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.853867 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.853868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.854724 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.854728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.855308 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.855309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.855362 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.855365 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.855449 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.855488 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260811 10:12:41.855952 22409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3319ae8e5743149552124d0f2f9e9d" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18" is_pre_election: true
I20260811 10:12:41.856137 22409 raft_consensus.cc:2454] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 0.
I20260811 10:12:41.856127 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3319ae8e5743149552124d0f2f9e9d" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123" is_pre_election: true
I20260811 10:12:41.856244 22644 raft_consensus.cc:2454] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 0.
I20260811 10:12:41.856335 22755 leader_election.cc:302] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, ca4cedda4ed94a6287e0f561b6611f47; no voters:
I20260811 10:12:41.856504 23003 raft_consensus.cc:2781] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:41.856559 23003 raft_consensus.cc:482] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:41.856580 23003 raft_consensus.cc:3037] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.857273 23003 raft_consensus.cc:504] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.857414 23003 leader_election.cc:288] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 election: Requested vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:41.857569 22409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3319ae8e5743149552124d0f2f9e9d" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18"
I20260811 10:12:41.857574 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3319ae8e5743149552124d0f2f9e9d" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123"
I20260811 10:12:41.857664 22644 raft_consensus.cc:3037] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.857668 22409 raft_consensus.cc:3037] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:41.858304 22644 raft_consensus.cc:2454] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 1.
I20260811 10:12:41.858492 22409 raft_consensus.cc:2454] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 1.
I20260811 10:12:41.858495 22752 leader_election.cc:302] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4cedda4ed94a6287e0f561b6611f47, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:41.858597 23003 raft_consensus.cc:2781] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:41.858713 23003 raft_consensus.cc:686] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 1 LEADER]: Becoming Leader. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Running, Role: LEADER
I20260811 10:12:41.858789 23003 consensus_queue.cc:237] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.859589 22154 catalog_manager.cc:5495] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3). New cstate: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } health_report { overall_health: HEALTHY } } }
Aug 11 10:12:41 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443160, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.62@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:41.892594 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.885327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.8.238.62:45573)
0811 10:12:41.885379 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:41.885381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:41.885391 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:41.885433 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:41.885434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:41.885446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:41.885520 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:41.886120 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.886904 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.887659 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.887937 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.888022 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.890127 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:41.890129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:41.890129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:41.890136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:41.891322 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.891707 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.891708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.891709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.891787 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:41.891988 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:41.891989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:41.891989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:41.892111 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:41.892183 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:41.892356 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 10:12:41.892517 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":25}
I20260811 10:12:41.892601 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:41.884580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45573 (local address 127.0.0.1:56898)
0811 10:12:41.884786 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:41.884800 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:41.884867 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:41.885570 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:41.885571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:41.885972 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:41.886022 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.886025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.886994 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.886996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.887572 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:41.887574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:41.888025 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:41.888026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:41.888082 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:41.888612 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:41.888617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:41.890035 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:41.891417 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.891425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.891427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.891630 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.891855 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:41.891856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:41.891856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:41.891923 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:41.892184 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:41.892192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:41.892240 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 10:12:41.892320 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:12:41.892463 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260811 10:12:41.894677 22154 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:56898:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 10:12:41.894904 22154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:41.896562 22371 tablet_service.cc:1467] Processing CreateTablet for tablet 79865daef9a244a59a18a9cb512764fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd9aa7b0b5134131918fbf4e1c9db2b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:41.896559 22838 tablet_service.cc:1467] Processing CreateTablet for tablet 79865daef9a244a59a18a9cb512764fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd9aa7b0b5134131918fbf4e1c9db2b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:41.896548 22608 tablet_service.cc:1467] Processing CreateTablet for tablet 79865daef9a244a59a18a9cb512764fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd9aa7b0b5134131918fbf4e1c9db2b5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 10:12:41.896746 22608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79865daef9a244a59a18a9cb512764fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.896742 22838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79865daef9a244a59a18a9cb512764fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.896756 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79865daef9a244a59a18a9cb512764fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:41.898147 23002 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: Bootstrap starting.
I20260811 10:12:41.898379 22996 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: Bootstrap starting.
I20260811 10:12:41.898389 22999 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: Bootstrap starting.
I20260811 10:12:41.898630 23002 tablet_bootstrap.cc:654] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.898734 22999 tablet_bootstrap.cc:654] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.898810 22996 tablet_bootstrap.cc:654] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:41.899384 22999 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: No bootstrap required, opened a new log
I20260811 10:12:41.899454 22999 ts_tablet_manager.cc:1397] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.899467 23002 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: No bootstrap required, opened a new log
I20260811 10:12:41.899511 23002 ts_tablet_manager.cc:1397] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:41.899586 22999 raft_consensus.cc:348] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.899673 22999 raft_consensus.cc:374] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.899662 22996 tablet_bootstrap.cc:492] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: No bootstrap required, opened a new log
I20260811 10:12:41.899696 22999 raft_consensus.cc:729] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.899665 23002 raft_consensus.cc:348] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.899751 23002 raft_consensus.cc:374] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.899734 22996 ts_tablet_manager.cc:1397] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:41.899776 23002 raft_consensus.cc:729] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.899768 22999 consensus_queue.cc:260] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.899823 23002 consensus_queue.cc:260] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.899878 22999 ts_tablet_manager.cc:1428] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:41.899937 23002 ts_tablet_manager.cc:1428] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:41.899928 22996 raft_consensus.cc:348] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.900010 22996 raft_consensus.cc:374] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:41.900038 22996 raft_consensus.cc:729] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Initialized, Role: FOLLOWER
I20260811 10:12:41.900005 23008 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:41.900075 23008 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:41.900089 22996 consensus_queue.cc:260] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:41.900061 23006 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:41.900159 23006 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:41.900192 22996 ts_tablet_manager.cc:1428] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:41.900336 23003 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:41.900426 23003 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:42.087881 23003 raft_consensus.cc:482] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:42.087986 23003 raft_consensus.cc:504] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.088194 23003 leader_election.cc:288] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:42.088445 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79865daef9a244a59a18a9cb512764fc" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123" is_pre_election: true
I20260811 10:12:42.088439 22409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79865daef9a244a59a18a9cb512764fc" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18" is_pre_election: true
I20260811 10:12:42.088554 22644 raft_consensus.cc:2454] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 0.
I20260811 10:12:42.088577 22409 raft_consensus.cc:2454] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 0.
I20260811 10:12:42.088716 22752 leader_election.cc:302] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4cedda4ed94a6287e0f561b6611f47, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.088821 23003 raft_consensus.cc:2781] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:42.088898 23003 raft_consensus.cc:482] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:42.088932 23003 raft_consensus.cc:3037] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.089744 23003 raft_consensus.cc:504] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.089912 23003 leader_election.cc:288] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 election: Requested vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111)
I20260811 10:12:42.090076 22409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79865daef9a244a59a18a9cb512764fc" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18"
I20260811 10:12:42.090097 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79865daef9a244a59a18a9cb512764fc" candidate_uuid: "ca4cedda4ed94a6287e0f561b6611f47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123"
I20260811 10:12:42.090163 22644 raft_consensus.cc:3037] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.090163 22409 raft_consensus.cc:3037] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.090986 22409 raft_consensus.cc:2454] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 1.
I20260811 10:12:42.090991 22644 raft_consensus.cc:2454] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca4cedda4ed94a6287e0f561b6611f47 in term 1.
I20260811 10:12:42.091181 22752 leader_election.cc:302] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4cedda4ed94a6287e0f561b6611f47, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.091296 23003 raft_consensus.cc:2781] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:42.091377 23003 raft_consensus.cc:686] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [term 1 LEADER]: Becoming Leader. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Running, Role: LEADER
I20260811 10:12:42.091430 23003 consensus_queue.cc:237] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.091724 23039 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: TxnStatusTablet state changed. Reason: New leader ca4cedda4ed94a6287e0f561b6611f47. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.091868 23039 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:42.092032 23064 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 10:12:42.092156 23064 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 10:12:42.092110 22154 catalog_manager.cc:5495] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3). New cstate: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } health_report { overall_health: HEALTHY } } }
I20260811 10:12:42.130065 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.126894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.0.0.1:45560)
0811 10:12:42.126994 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.127006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.127056 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.127301 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.127302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.127303 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:42.127368 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.127371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.128123 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.128124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.129097 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.129101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.129472 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.129473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.129539 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.129558 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.129847 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.129856 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.129913 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.129977 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 10:12:42.130131 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.126987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45560 (local address 127.8.238.3:36739)
0811 10:12:42.127126 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.127128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.127136 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:42.127181 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.127181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.127200 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.127282 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.127454 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.128045 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.129193 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.129403 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.129451 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.129673 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.129785 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.129837 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.129916 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.129935 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260811 10:12:42.131464 22644 raft_consensus.cc:1261] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Refusing update from remote peer ca4cedda4ed94a6287e0f561b6611f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.131561 22409 raft_consensus.cc:1261] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Refusing update from remote peer ca4cedda4ed94a6287e0f561b6611f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.131767 23039 consensus_queue.cc:1035] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.131901 23003 consensus_queue.cc:1035] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.132596 23006 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: TxnStatusTablet state changed. Reason: New leader ca4cedda4ed94a6287e0f561b6611f47. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.132699 23006 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:42.132661 23008 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: TxnStatusTablet state changed. Reason: New leader ca4cedda4ed94a6287e0f561b6611f47. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.132804 23008 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:42.133731 23003 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.133828 23003 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:42.133880 23003 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.133934 23003 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P ca4cedda4ed94a6287e0f561b6611f47: This TxnStatusTablet replica's current role is: LEADER
I20260811 10:12:42.133988 23070 mvcc.cc:205] Tried to move back new op lower bound from 7317271192088100864 to 7317271191926890496. Current Snapshot: MvccSnapshot[applied={T|T < 7317271192088100864}]
I20260811 10:12:42.134112 23008 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.134131 23006 tablet_replica.cc:439] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca4cedda4ed94a6287e0f561b6611f47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } } }
I20260811 10:12:42.134191 23008 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P c5f08c3946e9414d80db070344ab0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:42.134212 23006 tablet_replica.cc:442] T 79865daef9a244a59a18a9cb512764fc P e53d7e22ccf04435b2987f319db85123: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 10:12:42.139079 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.136479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.8.238.3:36739)
0811 10:12:42.136523 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.136525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.136645 (+ 120us) server_negotiation.cc:407] Connection header received
0811 10:12:42.136674 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.136675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.136686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.136753 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.136877 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.137613 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.138350 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.138645 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.138716 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.138850 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.138885 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.138937 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.138991 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.139011 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:42.144280 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.141567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:50075 (local address 127.8.238.3:36739)
0811 10:12:42.141607 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.141609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.141738 (+ 129us) server_negotiation.cc:407] Connection header received
0811 10:12:42.141766 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.141767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.141775 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.141837 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.141963 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.142646 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.143439 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.143729 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.143817 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.143982 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.144013 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.144064 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.144196 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.144217 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:12:42.144347 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.141509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.8.238.3:50075)
0811 10:12:42.141671 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.141680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.141731 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.141841 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.141841 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.141842 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:42.141906 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.141907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.142733 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.142734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.143358 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.143360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.143837 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.143838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.143888 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.143903 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.144103 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.144104 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.144165 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.144210 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260811 10:12:42.146492 22644 raft_consensus.cc:1261] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Refusing update from remote peer ca4cedda4ed94a6287e0f561b6611f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.146631 22409 raft_consensus.cc:1261] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Refusing update from remote peer ca4cedda4ed94a6287e0f561b6611f47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.146759 23079 consensus_queue.cc:1035] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.146911 23079 consensus_queue.cc:1035] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.158793 22152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.158903 22152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.159569 22152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Sending DeleteTablet for 3 replicas of tablet 7f3319ae8e5743149552124d0f2f9e9d
I20260811 10:12:42.159844 22371 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57688
I20260811 10:12:42.159863 22608 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47572
I20260811 10:12:42.160157 23100 tablet_replica.cc:330] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: stopping tablet replica
I20260811 10:12:42.160171 22836 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f3319ae8e5743149552124d0f2f9e9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 10:12:42.160219 23100 raft_consensus.cc:2229] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.160254 23100 raft_consensus.cc:2258] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.160382 23102 tablet_replica.cc:330] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: stopping tablet replica
I20260811 10:12:42.160436 23102 raft_consensus.cc:2229] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:42.160521 23102 raft_consensus.cc:2258] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.160775 23102 ts_tablet_manager.cc:1905] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.162593 23102 ts_tablet_manager.cc:1918] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:42.162663 23102 log.cc:1196] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/7f3319ae8e5743149552124d0f2f9e9d
I20260811 10:12:42.162897 23102 ts_tablet_manager.cc:1939] T 7f3319ae8e5743149552124d0f2f9e9d P ca4cedda4ed94a6287e0f561b6611f47: Deleting consensus metadata
I20260811 10:12:42.163208 22131 catalog_manager.cc:4841] TS ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739): tablet 7f3319ae8e5743149552124d0f2f9e9d (table test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]) successfully deleted
I20260811 10:12:42.163638 23101 tablet_replica.cc:330] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: stopping tablet replica
I20260811 10:12:42.163697 23100 ts_tablet_manager.cc:1905] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.163784 23101 raft_consensus.cc:2229] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.163825 23101 raft_consensus.cc:2258] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.165498 23100 ts_tablet_manager.cc:1918] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:42.165565 23100 log.cc:1196] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/7f3319ae8e5743149552124d0f2f9e9d
I20260811 10:12:42.165793 23100 ts_tablet_manager.cc:1939] T 7f3319ae8e5743149552124d0f2f9e9d P c5f08c3946e9414d80db070344ab0f18: Deleting consensus metadata
I20260811 10:12:42.166147 22133 catalog_manager.cc:4841] TS c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015): tablet 7f3319ae8e5743149552124d0f2f9e9d (table test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]) successfully deleted
I20260811 10:12:42.166549 23101 ts_tablet_manager.cc:1905] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.168341 23101 ts_tablet_manager.cc:1918] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:42.168414 23101 log.cc:1196] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/7f3319ae8e5743149552124d0f2f9e9d
I20260811 10:12:42.168654 23101 ts_tablet_manager.cc:1939] T 7f3319ae8e5743149552124d0f2f9e9d P e53d7e22ccf04435b2987f319db85123: Deleting consensus metadata
I20260811 10:12:42.169047 22131 catalog_manager.cc:4841] TS e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111): tablet 7f3319ae8e5743149552124d0f2f9e9d (table test-table [id=94bb259f6cd34cbd9a0759ea8bb6224b]) successfully deleted
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:12:42.188198 22233 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:42.182027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.8.238.62:45573)
0811 10:12:42.182080 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.182084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.182199 (+ 115us) server_negotiation.cc:407] Connection header received
0811 10:12:42.182275 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.182276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.182288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.182354 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:42.183146 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.183670 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.184505 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.184717 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.184836 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.188131 (+ 3295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56914: BlockingRecv error: recv got EOF from 127.0.0.1:56914 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM
I20260811 10:12:42.197481 22233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.189419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.8.238.62:45573)
0811 10:12:42.189457 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.189459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.189468 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:42.189538 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.189538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.189547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.189594 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:42.190231 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.190711 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.191828 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.192011 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.192066 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.194502 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:42.194504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:42.194505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:42.194513 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:42.196107 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:42.196577 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:42.196578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:42.196579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:42.196662 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:42.196887 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:42.196888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:42.196889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:42.197050 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:42.197159 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.197288 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.197372 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:12:42.199230 22152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:42.199431 22152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:42.201148 22371 tablet_service.cc:1467] Processing CreateTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 (DEFAULT_TABLE table=test-table [id=b913089b2a054014b59b5b8634804136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.201179 22836 tablet_service.cc:1467] Processing CreateTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 (DEFAULT_TABLE table=test-table [id=b913089b2a054014b59b5b8634804136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.201164 22608 tablet_service.cc:1467] Processing CreateTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 (DEFAULT_TABLE table=test-table [id=b913089b2a054014b59b5b8634804136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.201314 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe53c410a8c745eb934c90e1b1b5c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.201313 22608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe53c410a8c745eb934c90e1b1b5c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.201314 22836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe53c410a8c745eb934c90e1b1b5c827. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.202837 23002 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Bootstrap starting.
I20260811 10:12:42.202860 22999 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Bootstrap starting.
I20260811 10:12:42.202924 22996 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Bootstrap starting.
I20260811 10:12:42.203269 22999 tablet_bootstrap.cc:654] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.203339 23002 tablet_bootstrap.cc:654] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.203387 22996 tablet_bootstrap.cc:654] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.204090 22999 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: No bootstrap required, opened a new log
I20260811 10:12:42.204149 22999 ts_tablet_manager.cc:1397] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:42.204279 22999 raft_consensus.cc:348] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.204345 23002 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: No bootstrap required, opened a new log
I20260811 10:12:42.204366 22999 raft_consensus.cc:374] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.204397 22999 raft_consensus.cc:729] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.204393 23002 ts_tablet_manager.cc:1397] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:42.204448 22999 consensus_queue.cc:260] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.204548 22999 ts_tablet_manager.cc:1428] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 10:12:42.204555 23002 raft_consensus.cc:348] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.204618 23002 raft_consensus.cc:374] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.204635 23002 raft_consensus.cc:729] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.204676 23002 consensus_queue.cc:260] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.204778 23002 ts_tablet_manager.cc:1428] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:42.205338 22996 tablet_bootstrap.cc:492] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: No bootstrap required, opened a new log
I20260811 10:12:42.205394 22996 ts_tablet_manager.cc:1397] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:12:42.205590 22996 raft_consensus.cc:348] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.205691 22996 raft_consensus.cc:374] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.205720 22996 raft_consensus.cc:729] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.205772 22996 consensus_queue.cc:260] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.205873 22996 ts_tablet_manager.cc:1428] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:42.209125 23006 raft_consensus.cc:482] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:42.209198 23006 raft_consensus.cc:504] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.209518 23006 leader_election.cc:288] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739)
I20260811 10:12:42.213124 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.209884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:60489 (local address 127.8.238.3:36739)
0811 10:12:42.209932 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.209934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.209943 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:42.210017 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.210018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.210025 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.210093 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:42.210299 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.211074 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.212369 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.212756 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.212842 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.212856 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.213019 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.213063 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:12:42.213151 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.209681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.8.238.2:60489)
0811 10:12:42.209872 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.209883 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.209941 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.210115 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.210117 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.210120 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:42.210196 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.210198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.211182 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.211183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.212272 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.212275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.212869 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.212871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.212938 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.212942 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.213012 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.213056 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":54,"thread_start_us":110,"threads_started":1}
I20260811 10:12:42.213428 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe53c410a8c745eb934c90e1b1b5c827" candidate_uuid: "e53d7e22ccf04435b2987f319db85123" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47" is_pre_election: true
I20260811 10:12:42.213512 22872 raft_consensus.cc:2454] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e53d7e22ccf04435b2987f319db85123 in term 0.
I20260811 10:12:42.213675 22531 leader_election.cc:302] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca4cedda4ed94a6287e0f561b6611f47, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.213835 23006 raft_consensus.cc:2781] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:42.213896 23006 raft_consensus.cc:482] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:42.213919 23006 raft_consensus.cc:3037] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.214828 23006 raft_consensus.cc:504] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.214998 23006 leader_election.cc:288] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [CANDIDATE]: Term 1 election: Requested vote from peers c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015), ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739)
I20260811 10:12:42.215343 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.209681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34015 (local address 127.8.238.2:38259)
0811 10:12:42.210046 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.210054 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.210116 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.210259 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.210260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.210262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:12:42.210353 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.210355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.211239 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.211240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.212981 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.212985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.213784 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.213785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.215138 (+ 1353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.215143 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.215208 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.215252 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":35,"spinlock_wait_cycles":1920,"thread_start_us":112,"threads_started":1}
I20260811 10:12:42.215348 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.209762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:38259 (local address 127.8.238.1:34015)
0811 10:12:42.209816 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.209818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.210093 (+ 275us) server_negotiation.cc:407] Connection header received
0811 10:12:42.210176 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.210177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.210190 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.210256 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:12:42.210425 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.211129 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.213095 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.213661 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.213709 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.213719 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.215227 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.215280 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:42.215642 22409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe53c410a8c745eb934c90e1b1b5c827" candidate_uuid: "e53d7e22ccf04435b2987f319db85123" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18" is_pre_election: true
I20260811 10:12:42.215775 22409 raft_consensus.cc:2454] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e53d7e22ccf04435b2987f319db85123 in term 0.
I20260811 10:12:42.215853 22408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe53c410a8c745eb934c90e1b1b5c827" candidate_uuid: "e53d7e22ccf04435b2987f319db85123" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5f08c3946e9414d80db070344ab0f18"
I20260811 10:12:42.215919 22408 raft_consensus.cc:3037] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.216744 22408 raft_consensus.cc:2454] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e53d7e22ccf04435b2987f319db85123 in term 1.
I20260811 10:12:42.216974 22532 leader_election.cc:302] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.217084 23006 raft_consensus.cc:2781] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:42.217252 23006 raft_consensus.cc:686] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 1 LEADER]: Becoming Leader. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Running, Role: LEADER
I20260811 10:12:42.217357 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe53c410a8c745eb934c90e1b1b5c827" candidate_uuid: "e53d7e22ccf04435b2987f319db85123" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47"
I20260811 10:12:42.217358 23006 consensus_queue.cc:237] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.217427 22872 raft_consensus.cc:3037] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.218005 22154 catalog_manager.cc:5495] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 reported cstate change: term changed from 0 to 1, leader changed from <none> to e53d7e22ccf04435b2987f319db85123 (127.8.238.2). New cstate: current_term: 1 leader_uuid: "e53d7e22ccf04435b2987f319db85123" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:42.218243 22872 raft_consensus.cc:2454] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e53d7e22ccf04435b2987f319db85123 in term 1.
I20260811 10:12:42.223237 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.220519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.8.238.2:38111)
0811 10:12:42.220560 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.220562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.220571 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:42.220632 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.220633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.220651 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.220735 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.220820 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.221526 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.222385 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.222660 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.222766 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.222920 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.223039 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.223123 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.223154 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.223174 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:12:42.224125 22872 raft_consensus.cc:1261] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Refusing update from remote peer e53d7e22ccf04435b2987f319db85123: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.224162 22408 raft_consensus.cc:1261] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Refusing update from remote peer e53d7e22ccf04435b2987f319db85123: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.224413 23132 consensus_queue.cc:1035] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.224555 23006 consensus_queue.cc:1035] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.228467 22154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.228590 22154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.229470 22154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Sending DeleteTablet for 3 replicas of tablet fe53c410a8c745eb934c90e1b1b5c827
I20260811 10:12:42.229722 22608 tablet_service.cc:1514] Processing DeleteTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47572
I20260811 10:12:42.229769 22836 tablet_service.cc:1514] Processing DeleteTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 10:12:42.229874 23101 tablet_replica.cc:330] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: stopping tablet replica
I20260811 10:12:42.229954 23101 raft_consensus.cc:2229] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:42.230052 23101 raft_consensus.cc:2258] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.230194 23102 tablet_replica.cc:330] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: stopping tablet replica
I20260811 10:12:42.230268 23102 raft_consensus.cc:2229] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.230314 23102 raft_consensus.cc:2258] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.230405 22371 tablet_service.cc:1514] Processing DeleteTablet for tablet fe53c410a8c745eb934c90e1b1b5c827 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57688
I20260811 10:12:42.230492 23100 tablet_replica.cc:330] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: stopping tablet replica
I20260811 10:12:42.230547 23100 raft_consensus.cc:2229] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.230592 23100 raft_consensus.cc:2258] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.230654 23102 ts_tablet_manager.cc:1905] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.230934 23100 ts_tablet_manager.cc:1905] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.231103 22152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:12:42.231242 22152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:12:42.232344 23102 ts_tablet_manager.cc:1918] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:42.232398 23102 log.cc:1196] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/fe53c410a8c745eb934c90e1b1b5c827
I20260811 10:12:42.232587 23100 ts_tablet_manager.cc:1918] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:42.232641 23100 log.cc:1196] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/fe53c410a8c745eb934c90e1b1b5c827
I20260811 10:12:42.232620 23102 ts_tablet_manager.cc:1939] T fe53c410a8c745eb934c90e1b1b5c827 P ca4cedda4ed94a6287e0f561b6611f47: Deleting consensus metadata
I20260811 10:12:42.232847 23100 ts_tablet_manager.cc:1939] T fe53c410a8c745eb934c90e1b1b5c827 P c5f08c3946e9414d80db070344ab0f18: Deleting consensus metadata
I20260811 10:12:42.232823 22371 tablet_service.cc:1467] Processing CreateTablet for tablet 90aa8339df674abebc11fc37f4672d49 (DEFAULT_TABLE table=test-table [id=53069ef315d94f96844b7cc1d41f2985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.232954 22608 tablet_service.cc:1467] Processing CreateTablet for tablet 90aa8339df674abebc11fc37f4672d49 (DEFAULT_TABLE table=test-table [id=53069ef315d94f96844b7cc1d41f2985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.233026 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aa8339df674abebc11fc37f4672d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.233057 22608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aa8339df674abebc11fc37f4672d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.234220 22999 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: Bootstrap starting.
I20260811 10:12:42.234361 23002 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Bootstrap starting.
I20260811 10:12:42.232597 22836 tablet_service.cc:1467] Processing CreateTablet for tablet 90aa8339df674abebc11fc37f4672d49 (DEFAULT_TABLE table=test-table [id=53069ef315d94f96844b7cc1d41f2985]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:12:42.234537 22999 tablet_bootstrap.cc:654] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.234544 22836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90aa8339df674abebc11fc37f4672d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:42.234776 23002 tablet_bootstrap.cc:654] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.234898 22133 catalog_manager.cc:4841] TS c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015): tablet fe53c410a8c745eb934c90e1b1b5c827 (table test-table [id=b913089b2a054014b59b5b8634804136]) successfully deleted
I20260811 10:12:42.235517 23002 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: No bootstrap required, opened a new log
I20260811 10:12:42.235574 23002 ts_tablet_manager.cc:1397] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:42.235770 23002 raft_consensus.cc:348] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.235845 23002 raft_consensus.cc:374] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.235836 22131 catalog_manager.cc:4841] TS ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739): tablet fe53c410a8c745eb934c90e1b1b5c827 (table test-table [id=b913089b2a054014b59b5b8634804136]) successfully deleted
I20260811 10:12:42.235867 23002 raft_consensus.cc:729] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e53d7e22ccf04435b2987f319db85123, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.235910 23002 consensus_queue.cc:260] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.235948 22999 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: No bootstrap required, opened a new log
I20260811 10:12:42.236006 22999 ts_tablet_manager.cc:1397] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:12:42.236006 23002 ts_tablet_manager.cc:1428] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:42.236116 22999 raft_consensus.cc:348] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.236172 22996 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Bootstrap starting.
I20260811 10:12:42.236189 22999 raft_consensus.cc:374] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.236212 22999 raft_consensus.cc:729] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.236251 22999 consensus_queue.cc:260] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.236334 22999 ts_tablet_manager.cc:1428] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:42.236578 22996 tablet_bootstrap.cc:654] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:42.235528 23101 ts_tablet_manager.cc:1905] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.238289 23101 ts_tablet_manager.cc:1918] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:12:42.238343 23101 log.cc:1196] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/fe53c410a8c745eb934c90e1b1b5c827
I20260811 10:12:42.238551 23101 ts_tablet_manager.cc:1939] T fe53c410a8c745eb934c90e1b1b5c827 P e53d7e22ccf04435b2987f319db85123: Deleting consensus metadata
I20260811 10:12:42.238976 22131 catalog_manager.cc:4841] TS e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111): tablet fe53c410a8c745eb934c90e1b1b5c827 (table test-table [id=b913089b2a054014b59b5b8634804136]) successfully deleted
I20260811 10:12:42.240193 22996 tablet_bootstrap.cc:492] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: No bootstrap required, opened a new log
I20260811 10:12:42.240254 22996 ts_tablet_manager.cc:1397] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 10:12:42.240536 22996 raft_consensus.cc:348] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.240621 22996 raft_consensus.cc:374] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:42.240644 22996 raft_consensus.cc:729] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4cedda4ed94a6287e0f561b6611f47, State: Initialized, Role: FOLLOWER
I20260811 10:12:42.240688 22996 consensus_queue.cc:260] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.240794 22996 ts_tablet_manager.cc:1428] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:42.516181 23008 raft_consensus.cc:482] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:12:42.516320 23008 raft_consensus.cc:504] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.516563 23008 leader_election.cc:288] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111), ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739)
I20260811 10:12:42.516902 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aa8339df674abebc11fc37f4672d49" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123" is_pre_election: true
I20260811 10:12:42.517064 22644 raft_consensus.cc:2454] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 0.
I20260811 10:12:42.516990 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aa8339df674abebc11fc37f4672d49" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47" is_pre_election: true
I20260811 10:12:42.517164 22872 raft_consensus.cc:2454] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 0.
I20260811 10:12:42.517242 22298 leader_election.cc:302] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.517370 23008 raft_consensus.cc:2781] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:12:42.517431 23008 raft_consensus.cc:482] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:12:42.517467 23008 raft_consensus.cc:3037] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.518497 23008 raft_consensus.cc:504] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.518636 23008 leader_election.cc:288] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 election: Requested vote from peers e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111), ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739)
I20260811 10:12:42.518816 22644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aa8339df674abebc11fc37f4672d49" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e53d7e22ccf04435b2987f319db85123"
I20260811 10:12:42.518837 22872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90aa8339df674abebc11fc37f4672d49" candidate_uuid: "c5f08c3946e9414d80db070344ab0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca4cedda4ed94a6287e0f561b6611f47"
I20260811 10:12:42.518916 22872 raft_consensus.cc:3037] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.518930 22644 raft_consensus.cc:3037] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:42.519800 22644 raft_consensus.cc:2454] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 1.
I20260811 10:12:42.519801 22872 raft_consensus.cc:2454] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5f08c3946e9414d80db070344ab0f18 in term 1.
I20260811 10:12:42.519991 22298 leader_election.cc:302] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5f08c3946e9414d80db070344ab0f18, e53d7e22ccf04435b2987f319db85123; no voters:
I20260811 10:12:42.520097 23008 raft_consensus.cc:2781] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:42.520254 23008 raft_consensus.cc:686] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 1 LEADER]: Becoming Leader. State: Replica: c5f08c3946e9414d80db070344ab0f18, State: Running, Role: LEADER
I20260811 10:12:42.520370 23008 consensus_queue.cc:237] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } }
I20260811 10:12:42.520995 22154 catalog_manager.cc:5495] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5f08c3946e9414d80db070344ab0f18 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "c5f08c3946e9414d80db070344ab0f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5f08c3946e9414d80db070344ab0f18" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 } health_report { overall_health: UNKNOWN } } }
I20260811 10:12:42.594553 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.592327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57696 (local address 127.8.238.1:34015)
0811 10:12:42.592382 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.592384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.592392 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:42.592449 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.592449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.592459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.592506 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.592611 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.593166 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.593940 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.594132 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.594190 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.594352 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.594378 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.594416 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.594468 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.594492 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:12:42.599424 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.597161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39705 (local address 127.8.238.3:36739)
0811 10:12:42.597201 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.597203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.597323 (+ 120us) server_negotiation.cc:407] Connection header received
0811 10:12:42.597388 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.597389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.597400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.597456 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.597567 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.598079 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.598817 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.599021 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.599076 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.599223 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.599250 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.599288 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.599342 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.599365 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:12:42.599435 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.597101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:36739 (local address 127.8.238.1:39705)
0811 10:12:42.597261 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.597270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.597320 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.597476 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.597477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.597478 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:42.597518 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.597519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.598177 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.598178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.598740 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.598741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.599091 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.599092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.599132 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.599147 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.599303 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.599304 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.599338 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.599376 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260811 10:12:42.601686 22872 raft_consensus.cc:1261] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Refusing update from remote peer c5f08c3946e9414d80db070344ab0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.601725 22644 raft_consensus.cc:1261] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Refusing update from remote peer c5f08c3946e9414d80db070344ab0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:12:42.601967 23156 consensus_queue.cc:1035] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e53d7e22ccf04435b2987f319db85123" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.602196 23008 consensus_queue.cc:1035] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca4cedda4ed94a6287e0f561b6611f47" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 36739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:12:42.609284 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.606780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34015 (local address 127.8.238.3:57191)
0811 10:12:42.606829 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.606841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:42.606885 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:42.606999 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:42.607000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:42.607001 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 10:12:42.607046 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.607048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.607813 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.607814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.608352 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:42.608354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.608786 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.608787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:42.608845 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.608852 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.609087 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.609090 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:12:42.609160 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:12:42.609210 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 10:12:42.609279 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.606829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:57191 (local address 127.8.238.1:34015)
0811 10:12:42.606864 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:42.606866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:42.606874 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:42.606934 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:42.606934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:42.606944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:42.606993 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:12:42.607128 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.607693 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.608423 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:42.608710 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:42.608788 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:42.608933 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:12:42.608972 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:12:42.609035 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:42.609189 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 10:12:42.609210 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:12:42.615322 22154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.615427 22154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:12:42.616114 22154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff13f86463124373aa071b3862a646e8: Sending DeleteTablet for 3 replicas of tablet 90aa8339df674abebc11fc37f4672d49
I20260811 10:12:42.616379 22371 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aa8339df674abebc11fc37f4672d49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:57688
I20260811 10:12:42.616387 22608 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aa8339df674abebc11fc37f4672d49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47572
I20260811 10:12:42.616411 22836 tablet_service.cc:1514] Processing DeleteTablet for tablet 90aa8339df674abebc11fc37f4672d49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:12:42 UTC) from {username='oryx', principal='oryx/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 10:12:42.616482 23100 tablet_replica.cc:330] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: stopping tablet replica
I20260811 10:12:42.616513 23101 tablet_replica.cc:330] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: stopping tablet replica
I20260811 10:12:42.616544 23100 raft_consensus.cc:2229] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:12:42.616554 23102 tablet_replica.cc:330] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: stopping tablet replica
I20260811 10:12:42.616576 23101 raft_consensus.cc:2229] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.616612 23101 raft_consensus.cc:2258] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.616621 23100 raft_consensus.cc:2258] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.616628 23102 raft_consensus.cc:2229] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:12:42.616677 23102 raft_consensus.cc:2258] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:12:42.616920 23100 ts_tablet_manager.cc:1905] T 90aa8339df674abebc11fc37f4672d49 P c5f08c3946e9414d80db070344ab0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.616952 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 22267
I20260811 10:12:42.620136 23102 ts_tablet_manager.cc:1905] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.622488 23102 ts_tablet_manager.cc:1918] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:42.622577 23102 log.cc:1196] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/90aa8339df674abebc11fc37f4672d49
I20260811 10:12:42.622853 23102 ts_tablet_manager.cc:1939] T 90aa8339df674abebc11fc37f4672d49 P ca4cedda4ed94a6287e0f561b6611f47: Deleting consensus metadata
I20260811 10:12:42.623291 22131 catalog_manager.cc:4841] TS ca4cedda4ed94a6287e0f561b6611f47 (127.8.238.3:36739): tablet 90aa8339df674abebc11fc37f4672d49 (table test-table [id=53069ef315d94f96844b7cc1d41f2985]) successfully deleted
I20260811 10:12:42.624089 23101 ts_tablet_manager.cc:1905] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:12:42.626295 23101 ts_tablet_manager.cc:1918] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 10:12:42.626368 23101 log.cc:1196] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/90aa8339df674abebc11fc37f4672d49
I20260811 10:12:42.626601 23101 ts_tablet_manager.cc:1939] T 90aa8339df674abebc11fc37f4672d49 P e53d7e22ccf04435b2987f319db85123: Deleting consensus metadata
I20260811 10:12:42.632822 22131 catalog_manager.cc:4841] TS e53d7e22ccf04435b2987f319db85123 (127.8.238.2:38111): tablet 90aa8339df674abebc11fc37f4672d49 (table test-table [id=53069ef315d94f96844b7cc1d41f2985]) successfully deleted
W20260811 10:12:42.638406 22133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:34015 (error 108)
I20260811 10:12:42.638664 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 22507
I20260811 10:12:42.640730 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:42.638851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34015 (local address 127.0.0.1:57710)
0811 10:12:42.640617 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:42.640674 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1744,"thread_start_us":38,"threads_started":1}
W20260811 10:12:42.646214 22133 catalog_manager.cc:4584] TS c5f08c3946e9414d80db070344ab0f18 (127.8.238.1:34015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 90aa8339df674abebc11fc37f4672d49: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34015: connect: Connection refused (error 111)
I20260811 10:12:42.652128 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 22726
I20260811 10:12:42.663184 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 22100
2026-08-11T10:12:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3385 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[23178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[23178](info): set up 2 sockets
Aug 11 10:12:42 dist-test-slave-7f01 krb5kdc[23178](info): commencing operation
krb5kdc: starting...
W20260811 10:12:43.966501 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:44 dist-test-slave-7f01 krb5kdc[23178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:44.095068 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42471
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:33393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:44.181267 23201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:44.181478 23201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:44.181501 23201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:44.182588 23201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:44.182624 23201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:44.182639 23201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:44.182652 23201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:44.182664 23201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:44.184188 23201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:33393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:44.184386 23201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:44.184559 23201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:44.189540 23212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:44.189558 23209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:44.189551 23208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:44.189647 23201 server_base.cc:1034] running on GCE node
I20260811 10:12:44.190084 23201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:44.190414 23201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:44.191576 23201 hybrid_clock.cc:648] HybridClock initialized: now 1786443164191562 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T10:12:44Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1534 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:12:44 dist-test-slave-7f01 krb5kdc[23218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:12:44 dist-test-slave-7f01 krb5kdc[23218](info): set up 2 sockets
Aug 11 10:12:44 dist-test-slave-7f01 krb5kdc[23218](info): commencing operation
krb5kdc: starting...
W20260811 10:12:45.505416 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:12:45 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:12:45.636020 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44511
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:45.719619 23233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:45.719902 23233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:45.719944 23233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:45.721098 23233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:45.721171 23233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:45.721194 23233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:45.721215 23233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:45.721244 23233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:45.722721 23233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:45.723034 23233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:45.723280 23233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:45.728255 23243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:45.728276 23239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:45.728360 23240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:45.728600 23233 server_base.cc:1034] running on GCE node
I20260811 10:12:45.728866 23233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:45.729233 23233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:45.730409 23233 hybrid_clock.cc:648] HybridClock initialized: now 1786443165730400 us; error 43 us; skew 500 ppm
Aug 11 10:12:45 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443165, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:45.733150 23233 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:45.733582 23233 webserver.cc:460] Webserver started at http://127.8.238.62:43077/ using document root <none> and password file <none>
I20260811 10:12:45.733824 23233 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:45.733896 23233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:45.734056 23233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:45.735116 23233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "6d6ed007fef040369887307e9240e10d"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "876e82ac779ddc8133b707e50b9b863c"
server_key_iv: "76334831b4c0f0c01b440ae32f15f3aa"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.735503 23233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "6d6ed007fef040369887307e9240e10d"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "876e82ac779ddc8133b707e50b9b863c"
server_key_iv: "76334831b4c0f0c01b440ae32f15f3aa"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.736964 23233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:45.737766 23256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:45.737957 23233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:45.738075 23233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "6d6ed007fef040369887307e9240e10d"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "876e82ac779ddc8133b707e50b9b863c"
server_key_iv: "76334831b4c0f0c01b440ae32f15f3aa"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.738165 23233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:45.759979 23233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:45.762233 23233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:45.762418 23233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:45.767709 23233 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:44511
I20260811 10:12:45.767809 23350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:44511 every 8 connection(s)
I20260811 10:12:45.768193 23233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:45.769194 23352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:12:45.771306 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 23233
I20260811 10:12:45.771452 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:12:45.771641 9144 external_mini_cluster.cc:1428] Setting key ad44a8865db7f6ab199d2dcf21b1ac16
I20260811 10:12:45.771872 23352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Bootstrap starting.
I20260811 10:12:45.772709 23352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Neither blocks nor log segments found. Creating new log.
I20260811 10:12:45.773101 23352 log.cc:824] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:45.774080 23352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: No bootstrap required, opened a new log
I20260811 10:12:45.776608 23352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:45.776748 23352 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:12:45.776772 23352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6ed007fef040369887307e9240e10d, State: Initialized, Role: FOLLOWER
I20260811 10:12:45.776913 23352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:45.776996 23352 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:45.777024 23352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:45.777061 23352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:12:45.777765 23352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:45.777894 23352 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6ed007fef040369887307e9240e10d; no voters:
I20260811 10:12:45.778080 23352 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:12:45.778247 23360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:12:45.778363 23352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:45.778491 23360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 LEADER]: Becoming Leader. State: Replica: 6d6ed007fef040369887307e9240e10d, State: Running, Role: LEADER
I20260811 10:12:45.778651 23360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:45.779039 23360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6ed007fef040369887307e9240e10d. Latest consensus state: current_term: 1 leader_uuid: "6d6ed007fef040369887307e9240e10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } } }
I20260811 10:12:45.779085 23361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d6ed007fef040369887307e9240e10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } } }
I20260811 10:12:45.779203 23360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:45.779245 23361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: This master's current role is: LEADER
Aug 11 10:12:45 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:45.780594 23374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:45.780973 23374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:45.782503 23374 catalog_manager.cc:1347] Generated new cluster ID: cc7fb51094c44a8b80046b0ad34810ef
I20260811 10:12:45.782568 23374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:45.784296 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:45.772262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.8.238.62:44511)
0811 10:12:45.772501 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:45.772511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:45.772547 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:12:45.773230 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:45.773241 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:45.773311 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:45.773570 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:45.774904 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.776064 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.776850 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.777477 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.777603 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:45.780507 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:45.780510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:45.780511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:45.780521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:45.782638 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:45.783033 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:45.783037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:45.783037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:45.783152 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:45.783368 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:45.783369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:45.783369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:45.783680 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:45.783873 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:45.783953 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 10:12:45.784096 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260811 10:12:45.794283 23374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:12:45.795022 23374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:45.799381 23374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Generated new TSK 0
I20260811 10:12:45.799561 23374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:12:45.836160 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:45.933223 23392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:45.933461 23392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:45.933504 23392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:45.934696 23392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:45.934768 23392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:45.934823 23392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:45.936353 23392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:45.936615 23392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:45.936861 23392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:45.942179 23404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:45.942200 23400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:45.942241 23399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:45.942477 23392 server_base.cc:1034] running on GCE node
I20260811 10:12:45.942710 23392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:45.942989 23392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:45.944155 23392 hybrid_clock.cc:648] HybridClock initialized: now 1786443165944141 us; error 46 us; skew 500 ppm
Aug 11 10:12:45 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443165, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:45.946822 23392 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:12:45.947304 23392 webserver.cc:460] Webserver started at http://127.8.238.1:41531/ using document root <none> and password file <none>
I20260811 10:12:45.947551 23392 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:45.947624 23392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:45.947832 23392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:45.948891 23392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "f2f1454be37c4be48fe3f2c99e7138df"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "7c7847c1765adf69ec6463ff59fe7c6e"
server_key_iv: "474ef2a3a0169f93dc369ba0b581db55"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.949295 23392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "f2f1454be37c4be48fe3f2c99e7138df"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "7c7847c1765adf69ec6463ff59fe7c6e"
server_key_iv: "474ef2a3a0169f93dc369ba0b581db55"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.950660 23392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:45.951428 23414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:45.951601 23392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:45.951715 23392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "f2f1454be37c4be48fe3f2c99e7138df"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "7c7847c1765adf69ec6463ff59fe7c6e"
server_key_iv: "474ef2a3a0169f93dc369ba0b581db55"
server_key_version: "encryptionkey@0"
I20260811 10:12:45.951857 23392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:45.958281 23392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:45.960289 23392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:45.960451 23392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:45.960678 23392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:45.960992 23392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:45.961051 23392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:45.961092 23392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:45.961120 23392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:45.967463 23392 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:38697
I20260811 10:12:45.967511 23590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:38697 every 8 connection(s)
I20260811 10:12:45.967885 23392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:12:45 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443165, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:45.974314 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 23392
I20260811 10:12:45.974436 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:12:45.974677 9144 external_mini_cluster.cc:1428] Setting key 56526deb5c70f543c64e49d573d45644
I20260811 10:12:45.977914 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:45.968600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:43009 (local address 127.8.238.62:44511)
0811 10:12:45.968646 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:45.968648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:45.968813 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:12:45.969301 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:45.969303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:45.969314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:45.969377 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:45.970289 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.970805 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.971812 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.972060 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.972118 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:45.974427 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:45.974429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:45.974429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:45.974437 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:45.976406 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:45.976819 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:45.976820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:45.976821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:45.976893 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:45.977129 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:45.977130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:45.977130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:45.977283 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:45.977356 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:45.977684 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 10:12:45.977785 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:12:45.978062 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:45.968566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:43009)
0811 10:12:45.968714 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:45.968748 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:45.969209 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:45.969425 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:45.969428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:45.970009 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:45.970203 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:45.970209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.970889 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.970889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:45.971674 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:45.971676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:45.972150 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:45.972151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:45.972262 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:45.972804 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:45.972813 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:45.974310 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:45.976506 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:45.976513 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:45.976515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:45.976745 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:45.976961 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:45.976962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:45.976963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:45.977063 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:45.977372 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:45.977379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:45.977510 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 10:12:45.977655 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 10:12:45.977825 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260811 10:12:45.978680 23591 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44511
I20260811 10:12:45.978776 23591 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:45.978981 23591 heartbeater.cc:511] Master 127.8.238.62:44511 requested a full tablet report, sending...
I20260811 10:12:45.979640 23292 ts_manager.cc:194] Registered new tserver with Master: f2f1454be37c4be48fe3f2c99e7138df (127.8.238.1:38697)
I20260811 10:12:45.980331 23292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:43009
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:12:46.004092 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:46.097862 23598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.098073 23598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.098096 23598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.099310 23598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.099350 23598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.099376 23598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:46.100795 23598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.101038 23598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.101285 23598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.106328 23611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.106413 23606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.106559 23598 server_base.cc:1034] running on GCE node
W20260811 10:12:46.106601 23604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.106842 23598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.107043 23598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.108227 23598 hybrid_clock.cc:648] HybridClock initialized: now 1786443166108199 us; error 37 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.110889 23598 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:12:46.111367 23598 webserver.cc:460] Webserver started at http://127.8.238.2:35243/ using document root <none> and password file <none>
I20260811 10:12:46.111605 23598 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:46.111651 23598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:46.111837 23598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:46.112883 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "fa7175b4b40b40a0b6deb05597e74fbb"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "d80b8f40055cab31e4239f40ceabd5c7"
server_key_iv: "cf74993c0751b2e50a25e48e929acace"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.113250 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "fa7175b4b40b40a0b6deb05597e74fbb"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "d80b8f40055cab31e4239f40ceabd5c7"
server_key_iv: "cf74993c0751b2e50a25e48e929acace"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.114642 23598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:12:46.115447 23626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.115658 23598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:46.115785 23598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "fa7175b4b40b40a0b6deb05597e74fbb"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "d80b8f40055cab31e4239f40ceabd5c7"
server_key_iv: "cf74993c0751b2e50a25e48e929acace"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.115864 23598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:46.132503 23598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:46.134475 23598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:46.134591 23598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:46.134775 23598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:46.135133 23598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:46.135170 23598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.135192 23598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:46.135241 23598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.141490 23598 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:45491
I20260811 10:12:46.141573 23823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:45491 every 8 connection(s)
I20260811 10:12:46.141907 23598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:46.149685 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 23598
I20260811 10:12:46.149802 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:12:46.150045 9144 external_mini_cluster.cc:1428] Setting key f221a56a2f76811bce09b56ae481ffed
I20260811 10:12:46.151557 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.142653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:59601 (local address 127.8.238.62:44511)
0811 10:12:46.142723 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.142725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.142962 (+ 237us) server_negotiation.cc:407] Connection header received
0811 10:12:46.143462 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.143463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.143473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.143527 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.144540 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.145057 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.146026 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.146212 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.146261 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.148423 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:46.148424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:46.148425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:46.148432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:46.149996 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:46.150502 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:46.150503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:46.150503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:46.150569 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:46.150871 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:46.150872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:46.150873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:46.151019 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:46.151075 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:46.151415 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 10:12:46.151491 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:12:46.151902 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.142643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:59601)
0811 10:12:46.142840 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.142876 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.143376 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.143583 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.143586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.144254 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.144457 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.144462 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.145143 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.145144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.145950 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.145952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.146286 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.146287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.146388 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.146849 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.146858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.148330 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:46.150073 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:46.150081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:46.150083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:46.150408 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:46.150636 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:46.150636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:46.150636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:46.150801 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:46.151095 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:46.151103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:46.151274 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 10:12:46.151393 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 10:12:46.151590 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260811 10:12:46.152311 23824 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44511
I20260811 10:12:46.152396 23824 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:46.152593 23824 heartbeater.cc:511] Master 127.8.238.62:44511 requested a full tablet report, sending...
I20260811 10:12:46.153067 23292 ts_manager.cc:194] Registered new tserver with Master: fa7175b4b40b40a0b6deb05597e74fbb (127.8.238.2:45491)
I20260811 10:12:46.153462 23292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:59601
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:12:46.180243 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:46.272857 23830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.273085 23830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.273109 23830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.274214 23830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.274250 23830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.274276 23830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:46.275650 23830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.275890 23830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.276085 23830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.281159 23841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.281167 23838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.281167 23839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.282794 23830 server_base.cc:1034] running on GCE node
I20260811 10:12:46.283025 23830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.283277 23830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.284466 23830 hybrid_clock.cc:648] HybridClock initialized: now 1786443166284445 us; error 51 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.287091 23830 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:12:46.287585 23830 webserver.cc:460] Webserver started at http://127.8.238.3:46205/ using document root <none> and password file <none>
I20260811 10:12:46.287881 23830 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:46.287952 23830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:46.288106 23830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:12:46.289167 23830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "bf424e73bf0a4545bf92bacc18106240"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "b3f6cb9baf786f59a523ea94f40cee1c"
server_key_iv: "df594c78e7f7b168c7bdd02a393690d0"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.289553 23830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "bf424e73bf0a4545bf92bacc18106240"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "b3f6cb9baf786f59a523ea94f40cee1c"
server_key_iv: "df594c78e7f7b168c7bdd02a393690d0"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.290952 23830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:12:46.291790 23855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.291983 23830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:46.292093 23830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "bf424e73bf0a4545bf92bacc18106240"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "b3f6cb9baf786f59a523ea94f40cee1c"
server_key_iv: "df594c78e7f7b168c7bdd02a393690d0"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.292178 23830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:46.311304 23830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:46.313211 23830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:46.313385 23830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:46.313632 23830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:46.313951 23830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:46.314009 23830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.314054 23830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:46.314076 23830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.320634 23830 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:41519
I20260811 10:12:46.320670 24031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:41519 every 8 connection(s)
I20260811 10:12:46.321012 23830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:12:46.329468 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 23830
I20260811 10:12:46.329589 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:12:46.329780 9144 external_mini_cluster.cc:1428] Setting key 99dce1b1855245738f09c0bede26c436
I20260811 10:12:46.330338 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.321760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:60539 (local address 127.8.238.62:44511)
0811 10:12:46.321844 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.321846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.322030 (+ 184us) server_negotiation.cc:407] Connection header received
0811 10:12:46.322494 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.322495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.322507 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.322561 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.323492 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.324078 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.325187 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.325373 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.325430 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.327560 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:46.327562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:46.327562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:46.327573 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:46.328891 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:46.329262 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:46.329263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:46.329264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:46.329321 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:46.329631 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:46.329632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:46.329632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:46.329789 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:46.329863 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:46.330160 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 10:12:46.330243 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 10:12:46.330545 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.321712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:60539)
0811 10:12:46.321899 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.321935 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.322413 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.322614 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.322617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.323201 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.323413 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.323418 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.324158 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.324159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.325022 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.325024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.325454 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.325455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.325555 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.326092 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.326100 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.327478 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:12:46.328964 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:46.328979 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:46.328981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:46.329194 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:46.329416 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:12:46.329417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:12:46.329417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:12:46.329565 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:12:46.329866 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:12:46.329874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:12:46.330021 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 10:12:46.330132 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 10:12:46.330292 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":108,"threads_started":1}
I20260811 10:12:46.331025 24032 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44511
I20260811 10:12:46.331133 24032 heartbeater.cc:464] Registering TS with master...
I20260811 10:12:46.331313 24032 heartbeater.cc:511] Master 127.8.238.62:44511 requested a full tablet report, sending...
I20260811 10:12:46.331705 23292 ts_manager.cc:194] Registered new tserver with Master: bf424e73bf0a4545bf92bacc18106240 (127.8.238.3:41519)
I20260811 10:12:46.332120 23292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:60539
I20260811 10:12:46.341513 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
Entry for principal oryx/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab.
WARNING: no policy specified for oryx/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
Entry for principal oryx/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab.
WARNING: no policy specified for oryx/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
Entry for principal oryx/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab.
I20260811 10:12:46.425976 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 23392
I20260811 10:12:46.447824 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 23598
I20260811 10:12:46.465286 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 23830
I20260811 10:12:46.480717 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 23233
I20260811 10:12:46.496928 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44511
--webserver_interface=127.8.238.62
--webserver_port=43077
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:12:46.580790 24049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.581004 24049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.581027 24049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.582211 24049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:12:46.582257 24049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.582271 24049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.582312 24049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:12:46.582340 24049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:12:46.583861 24049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.584120 24049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.584406 24049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.589319 24058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.589311 24064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.589311 24060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.589545 24049 server_base.cc:1034] running on GCE node
I20260811 10:12:46.589866 24049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.590212 24049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.591385 24049 hybrid_clock.cc:648] HybridClock initialized: now 1786443166591363 us; error 37 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.593966 24049 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:12:46.594444 24049 webserver.cc:460] Webserver started at http://127.8.238.62:43077/ using document root <none> and password file <none>
I20260811 10:12:46.594671 24049 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:46.594755 24049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:46.596180 24049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:12:46.596868 24072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.597048 24049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:12:46.597162 24049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "6d6ed007fef040369887307e9240e10d"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "876e82ac779ddc8133b707e50b9b863c"
server_key_iv: "76334831b4c0f0c01b440ae32f15f3aa"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.597509 24049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:46.610702 24049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:46.612694 24049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:46.612861 24049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:46.617393 24049 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:44511
I20260811 10:12:46.617426 24150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:44511 every 8 connection(s)
I20260811 10:12:46.617828 24049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:12:46.619468 24151 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:12:46.620594 24151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Bootstrap starting.
I20260811 10:12:46.622198 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 24049
I20260811 10:12:46.622555 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:38697
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:44511
--webserver_port=41531
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:12:46.631322 24151 log.cc:824] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:12:46.633311 24151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:12:46.633648 24151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Bootstrap complete.
I20260811 10:12:46.636202 24151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:46.636445 24151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6ed007fef040369887307e9240e10d, State: Initialized, Role: FOLLOWER
I20260811 10:12:46.636627 24151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:46.636703 24151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:12:46.636765 24151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:12:46.636817 24151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:12:46.637970 24151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:46.638095 24151 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6ed007fef040369887307e9240e10d; no voters:
I20260811 10:12:46.638320 24151 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:12:46.638362 24158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:12:46.638540 24158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [term 2 LEADER]: Becoming Leader. State: Replica: 6d6ed007fef040369887307e9240e10d, State: Running, Role: LEADER
I20260811 10:12:46.638623 24151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:12:46.638708 24158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } }
I20260811 10:12:46.639053 24162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6ed007fef040369887307e9240e10d. Latest consensus state: current_term: 2 leader_uuid: "6d6ed007fef040369887307e9240e10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } } }
I20260811 10:12:46.639081 24161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6d6ed007fef040369887307e9240e10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6ed007fef040369887307e9240e10d" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44511 } } }
I20260811 10:12:46.639158 24162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:46.639169 24161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d [sys.catalog]: This master's current role is: LEADER
I20260811 10:12:46.639389 24167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:12:46.640040 24167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:12:46.640281 24167 catalog_manager.cc:1259] Loaded cluster ID: cc7fb51094c44a8b80046b0ad34810ef
I20260811 10:12:46.640419 24167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:12:46.641721 24167 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:12:46.641891 24167 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6d6ed007fef040369887307e9240e10d: Loaded TSK: 0
I20260811 10:12:46.642102 24167 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:12:46.709758 24155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.709971 24155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.709995 24155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.711191 24155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.711236 24155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.711262 24155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:12:46.712800 24155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.1.keytab
--principal=oryx/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:38697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=41531
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.713042 24155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.713297 24155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.719384 24195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.719455 24194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.719386 24197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.719655 24155 server_base.cc:1034] running on GCE node
I20260811 10:12:46.719959 24155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.720242 24155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.721431 24155 hybrid_clock.cc:648] HybridClock initialized: now 1786443166721413 us; error 36 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.724305 24155 init.cc:377] Logged in from keytab as oryx/127.8.238.1@KRBTEST.COM (short username oryx)
I20260811 10:12:46.724891 24155 webserver.cc:460] Webserver started at http://127.8.238.1:41531/ using document root <none> and password file <none>
I20260811 10:12:46.725207 24155 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:46.725334 24155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:46.727154 24155 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:46.727890 24211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.728092 24155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:12:46.728199 24155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "f2f1454be37c4be48fe3f2c99e7138df"
format_stamp: "Formatted at 2026-08-11 10:12:45 on dist-test-slave-7f01"
server_key: "7c7847c1765adf69ec6463ff59fe7c6e"
server_key_iv: "474ef2a3a0169f93dc369ba0b581db55"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.728688 24155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:46.739719 24155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:46.742394 24155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:46.742551 24155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:46.742810 24155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:46.743172 24155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:46.743229 24155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.743264 24155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:46.743291 24155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.751397 24155 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:38697
I20260811 10:12:46.751430 24399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:38697 every 8 connection(s)
I20260811 10:12:46.751884 24155 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:12:46.759147 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 24155
I20260811 10:12:46.759465 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:45491
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:44511
--webserver_port=35243
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:46.762086 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.752601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:44205)
0811 10:12:46.752826 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.752875 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.753524 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.753975 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.753979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.754750 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.755027 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.755034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.756372 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.756373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.757603 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.757607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.758243 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.758244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.758401 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.758947 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.758958 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.761885 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":124,"threads_started":1}
W20260811 10:12:46.762264 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.752771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:44205 (local address 127.8.238.62:44511)
0811 10:12:46.752945 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.752953 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.752989 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:12:46.753732 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.753738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.753780 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.753914 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.755188 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.756251 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.757711 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.758134 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.758234 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.762051 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:44205: BlockingRecv error: recv got EOF from 127.8.238.1:44205 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
W20260811 10:12:46.762626 24400 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:44511 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.238.62:44511: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:46.767364 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.762868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:46093)
0811 10:12:46.762923 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.762934 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.762978 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.763143 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.763145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.763274 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.763317 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.763318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.764012 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.764013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.764513 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.764515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.764886 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.764887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.764963 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.765445 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.765449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.767274 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 10:12:46.767416 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.762923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:46093 (local address 127.8.238.62:44511)
0811 10:12:46.762963 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.762965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.762975 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:46.763042 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.763043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.763055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.763128 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.763411 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.763933 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.764610 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.764810 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.764888 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.767350 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:46093: BlockingRecv error: recv got EOF from 127.8.238.1:46093 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:46.771955 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.767874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:38409)
0811 10:12:46.767948 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.767958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.768004 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.768147 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.768148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.768286 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.768310 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.768311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.768913 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.768914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.769403 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.769404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.769768 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.769769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.769824 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.770226 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.770231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.771879 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 10:12:46.772012 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.767911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:38409 (local address 127.8.238.62:44511)
0811 10:12:46.767948 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.767950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.767990 (+ 40us) server_negotiation.cc:407] Connection header received
0811 10:12:46.768059 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.768060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.768069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.768141 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.768391 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.768827 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.769499 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.769683 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.769766 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.771949 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:38409: BlockingRecv error: recv got EOF from 127.8.238.1:38409 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260811 10:12:46.772313 24400 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:12:46.853747 24406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.853971 24406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.853995 24406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.855098 24406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.855130 24406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.855155 24406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:12:46.856662 24406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.2.keytab
--principal=oryx/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:45491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=35243
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.856899 24406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.857089 24406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.862354 24413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.862421 24417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.862421 24414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.862774 24406 server_base.cc:1034] running on GCE node
I20260811 10:12:46.863112 24406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.863368 24406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.864521 24406 hybrid_clock.cc:648] HybridClock initialized: now 1786443166864496 us; error 42 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.867194 24406 init.cc:377] Logged in from keytab as oryx/127.8.238.2@KRBTEST.COM (short username oryx)
I20260811 10:12:46.867594 24406 webserver.cc:460] Webserver started at http://127.8.238.2:35243/ using document root <none> and password file <none>
I20260811 10:12:46.867838 24406 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:46.867899 24406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:46.869266 24406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:46.869962 24428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.870234 24406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:12:46.870352 24406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "fa7175b4b40b40a0b6deb05597e74fbb"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "d80b8f40055cab31e4239f40ceabd5c7"
server_key_iv: "cf74993c0751b2e50a25e48e929acace"
server_key_version: "encryptionkey@0"
I20260811 10:12:46.870721 24406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:46.878011 24406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:46.880329 24406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:46.880502 24406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:46.880741 24406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:46.881049 24406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:46.881105 24406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.881145 24406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:46.881171 24406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:46.887540 24406 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:45491
I20260811 10:12:46.887645 24619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:45491 every 8 connection(s)
I20260811 10:12:46.887985 24406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:12:46.895969 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.888773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:42787 (local address 127.8.238.62:44511)
0811 10:12:46.888885 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.888887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.889067 (+ 180us) server_negotiation.cc:407] Connection header received
0811 10:12:46.889539 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.889540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.889557 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.889617 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.890584 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.891121 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.892209 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.892426 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.892492 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.895916 (+ 3424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:42787: BlockingRecv error: recv got EOF from 127.8.238.2:42787 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:12:46.895993 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.888720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:42787)
0811 10:12:46.888915 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.888951 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.889442 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.889697 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.889701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.890276 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.890471 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.890477 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.891215 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.891215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.892112 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.892115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.892510 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.892510 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.892610 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.893060 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.893069 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.895839 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
W20260811 10:12:46.896253 24621 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:44511 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.238.62:44511: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
I20260811 10:12:46.897022 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 24406
I20260811 10:12:46.897320 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:41519
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:44511
--webserver_port=46205
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:46.900915 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.896477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:43305)
0811 10:12:46.896530 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.896541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.896606 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.896747 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.896748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.896883 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.896929 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.896931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.897679 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.897680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.898176 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.898178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.898545 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.898546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.898617 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.899066 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.899073 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.900870 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:12:46.901113 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.896568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43305 (local address 127.8.238.62:44511)
0811 10:12:46.896629 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.896631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.896639 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:46.896664 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.896665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.896674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.896745 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.897013 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.897603 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.898269 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.898468 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.898535 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.901030 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:43305: BlockingRecv error: recv got EOF from 127.8.238.2:43305 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:46.905886 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:46.901321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:38345)
0811 10:12:46.901367 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:46.901378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:46.901422 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:46.901543 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:46.901544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:46.901682 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:46.901712 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.901714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.902628 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.902629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.903092 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:46.903093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.903580 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.903581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:46.903617 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.904083 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:46.904087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:46.905843 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 10:12:46.906037 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:46.901371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:38345 (local address 127.8.238.62:44511)
0811 10:12:46.901404 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:46.901406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:46.901414 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:46.901473 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:46.901474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:46.901484 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:46.901542 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:46.901784 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.902549 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.903174 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:46.903498 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:46.903568 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:46.905961 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:38345: BlockingRecv error: recv got EOF from 127.8.238.2:38345 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260811 10:12:46.906064 24621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:12:46.985419 24629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:12:46.985674 24629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:12:46.985721 24629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:12:46.987206 24629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:12:46.987293 24629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:12:46.987339 24629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:12:46.988940 24629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/krb5kdc/oryx_127.8.238.3.keytab
--principal=oryx/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:41519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=46205
--tserver_master_addrs=127.8.238.62:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:12:46.989238 24629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:12:46.989487 24629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:12:46.994952 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:12:46.995038 24641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.995119 24629 server_base.cc:1034] running on GCE node
W20260811 10:12:46.994961 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:12:46.995434 24629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:12:46.995641 24629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:12:46.996802 24629 hybrid_clock.cc:648] HybridClock initialized: now 1786443166996797 us; error 36 us; skew 500 ppm
Aug 11 10:12:46 dist-test-slave-7f01 krb5kdc[23218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443166, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:12:46.999637 24629 init.cc:377] Logged in from keytab as oryx/127.8.238.3@KRBTEST.COM (short username oryx)
I20260811 10:12:47.000104 24629 webserver.cc:460] Webserver started at http://127.8.238.3:46205/ using document root <none> and password file <none>
I20260811 10:12:47.000391 24629 fs_manager.cc:359] Metadata directory not provided
I20260811 10:12:47.000495 24629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:12:47.001906 24629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:12:47.002569 24652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:47.002760 24629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:12:47.002857 24629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "bf424e73bf0a4545bf92bacc18106240"
format_stamp: "Formatted at 2026-08-11 10:12:46 on dist-test-slave-7f01"
server_key: "b3f6cb9baf786f59a523ea94f40cee1c"
server_key_iv: "df594c78e7f7b168c7bdd02a393690d0"
server_key_version: "encryptionkey@0"
I20260811 10:12:47.003216 24629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:12:47.014562 24629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:12:47.016561 24629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:12:47.016718 24629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:12:47.016952 24629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:12:47.017272 24629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:12:47.017338 24629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:47.017372 24629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:12:47.017400 24629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:12:47.023171 24629 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:41519
I20260811 10:12:47.023272 24829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:41519 every 8 connection(s)
I20260811 10:12:47.023626 24629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:12:47.024554 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 24629
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 10:12:47.032618 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:47.024425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:53353 (local address 127.8.238.62:44511)
0811 10:12:47.024507 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.024510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.024684 (+ 174us) server_negotiation.cc:407] Connection header received
0811 10:12:47.025495 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.025496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.025505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.025546 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.026730 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.027468 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.028880 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.029160 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.029251 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.032544 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:53353: BlockingRecv error: recv got EOF from 127.8.238.3:53353 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:12:47.032691 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.024382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:53353)
0811 10:12:47.024557 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:47.024598 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:47.025413 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:47.025619 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:47.025623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:47.026359 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:47.026618 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.026624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.027570 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.027571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.028754 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.028758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.029289 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.029290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.029419 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.029972 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:47.029980 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:47.032441 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
W20260811 10:12:47.032933 24831 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:44511 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.238.62:44511: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
I20260811 10:12:47.033975 24839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.025099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.8.238.62:44511)
0811 10:12:47.025241 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.025243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.025256 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:12:47.025291 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.025292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.025307 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.025388 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.025987 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.026805 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.027873 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.028189 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.028268 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.029751 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:12:47.029756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:12:47.029756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:12:47.029770 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:12:47.032014 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:47.032533 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:47.032537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:47.032537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:47.032686 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:12:47.032957 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:12:47.032958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:12:47.032958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:12:47.033313 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:12:47.033472 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 10:12:47.033637 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 10:12:47.033760 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:47.037750 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.033176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:56697)
0811 10:12:47.033297 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:47.033307 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:47.033362 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:47.033528 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:47.033529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:47.033664 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:47.033712 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.033713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.034565 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.034566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.035243 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.035252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.035571 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.035572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.035621 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.036010 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:47.036014 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:47.037704 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 10:12:47.037875 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:47.033235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:56697 (local address 127.8.238.62:44511)
0811 10:12:47.033273 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.033275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.033358 (+ 83us) server_negotiation.cc:407] Connection header received
0811 10:12:47.033457 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.033458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.033469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.033513 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.033802 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.034446 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.035322 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.035505 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.035555 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.037801 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:56697: BlockingRecv error: recv got EOF from 127.8.238.3:56697 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:47.041927 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.038047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:35237)
0811 10:12:47.038081 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:47.038091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:47.038136 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:47.038280 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:47.038281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:47.038396 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:47.038422 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.038423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.039021 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.039022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.039580 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.039582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.039942 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.039943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.040008 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.040317 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:47.040320 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:47.041878 (+ 1558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260811 10:12:47.042084 24405 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:47.038130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:35237 (local address 127.8.238.62:44511)
0811 10:12:47.038185 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.038187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.038195 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:47.038215 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.038216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.038224 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.038278 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.038479 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.038913 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.039653 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.039876 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.039932 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.041982 (+ 2050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:35237: BlockingRecv error: recv got EOF from 127.8.238.3:35237 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 10:12:47.042131 24831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:47.778040 24840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.772773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:47033)
0811 10:12:47.772936 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:47.772978 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:47.773085 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:47.773309 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:47.773310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:47.773541 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:47.773606 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.773608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.774455 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.774456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.775088 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.775091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.775486 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.775488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.775551 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.776060 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:47.776064 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:47.777924 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
W20260811 10:12:47.778096 24842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:47.772862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:47033 (local address 127.8.238.62:44511)
0811 10:12:47.773025 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.773027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.773040 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:12:47.773161 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.773162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.773178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.773249 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.773701 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.774385 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.775184 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.775405 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.775485 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.778017 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:47033: BlockingRecv error: recv got EOF from 127.8.238.1:47033 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:47 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:47.911823 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:47.906422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:34955)
0811 10:12:47.906583 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:47.906597 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:47.906675 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:47.906842 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:47.906844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:47.907041 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:47.907096 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.907098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.907855 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.907856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.908431 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:47.908433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.908895 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.908896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:47.908958 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.909494 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:47.909500 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:47.911621 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
W20260811 10:12:47.911827 24842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:47.906466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:34955 (local address 127.8.238.62:44511)
0811 10:12:47.906550 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:47.906553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:47.906670 (+ 117us) server_negotiation.cc:407] Connection header received
0811 10:12:47.906741 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:47.906743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:47.906771 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:47.906831 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:47.907198 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.907784 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.908539 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:47.908817 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:47.908896 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:47.911768 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:34955: BlockingRecv error: recv got EOF from 127.8.238.2:34955 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:48.047381 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:48.042526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:58851)
0811 10:12:48.042712 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:48.042722 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:48.042780 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:48.042892 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:48.042893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:48.043077 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:48.043140 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.043142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.043846 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.043847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.044403 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.044405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.044735 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.044736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.044777 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.045338 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:48.045343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:48.047260 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":124,"threads_started":1}
W20260811 10:12:48.047422 24842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:48.042574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:58851 (local address 127.8.238.62:44511)
0811 10:12:48.042644 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:48.042646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:48.042784 (+ 138us) server_negotiation.cc:407] Connection header received
0811 10:12:48.042824 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:48.042824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:48.042835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:48.042886 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:48.043231 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.043779 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.044478 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.044666 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.044719 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.047350 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:58851: BlockingRecv error: recv got EOF from 127.8.238.3:58851 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:48.783093 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:48.778517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:33035)
0811 10:12:48.778684 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:48.778696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:48.778755 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:48.778907 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:48.778909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:48.779090 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:48.779147 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.779148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.779795 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.779796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.780334 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.780336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.780660 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.780661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.780703 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.781205 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:48.781210 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:48.783030 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
W20260811 10:12:48.783214 24850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:48.778592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:33035 (local address 127.8.238.62:44511)
0811 10:12:48.778697 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:48.778699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:48.778742 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:12:48.778808 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:48.778809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:48.778820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:48.778870 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:48.779214 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.779722 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.780405 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.780597 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.780647 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.783129 (+ 2482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:33035: BlockingRecv error: recv got EOF from 127.8.238.1:33035 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":53,"threads_started":1}
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:48 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:48.917394 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:48.912637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:46153)
0811 10:12:48.912804 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:48.912814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:48.912890 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:48.913026 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:48.913027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:48.913207 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:48.913262 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.913264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.913920 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.913921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.914455 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:48.914456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.914793 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.914794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:48.914836 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.915328 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:48.915333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:48.917324 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
W20260811 10:12:48.917500 24850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:48.912731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:46153 (local address 127.8.238.62:44511)
0811 10:12:48.912803 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:48.912807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:48.912863 (+ 56us) server_negotiation.cc:407] Connection header received
0811 10:12:48.912958 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:48.912959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:48.912970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:48.913021 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:48.913329 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.913838 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.914534 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:48.914725 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:48.914781 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:48.917427 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:46153: BlockingRecv error: recv got EOF from 127.8.238.2:46153 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:49.053207 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:49.048039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:58103)
0811 10:12:49.048239 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:49.048260 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:49.048351 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:49.048494 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:49.048495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:49.048700 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:49.048752 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.048754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.049705 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.049706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.050324 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.050328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.050676 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.050676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.050722 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.051164 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:49.051168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:49.053097 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
W20260811 10:12:49.053241 24850 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:49.048282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:58103 (local address 127.8.238.62:44511)
0811 10:12:49.048352 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:49.048354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:49.048364 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:49.048418 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:49.048419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:49.048434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:49.048488 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:49.048862 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.049614 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.050413 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.050599 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.050656 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.053173 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:58103: BlockingRecv error: recv got EOF from 127.8.238.3:58103 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:49.793365 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:49.783637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:50903)
0811 10:12:49.783813 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:49.783828 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:49.783899 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:49.784110 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:49.784112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:49.784319 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:49.784420 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.784423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.785264 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.785275 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.785839 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.785840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.786211 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.786212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.786258 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.791230 (+ 4972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:49.791235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:49.793287 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
W20260811 10:12:49.793488 24854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:49.783773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:50903 (local address 127.8.238.62:44511)
0811 10:12:49.783893 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:49.783895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:49.783905 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:49.783979 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:49.783980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:49.783992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:49.784057 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:49.784526 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.785184 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.785930 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.786136 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.786194 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.793399 (+ 7205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:50903: BlockingRecv error: recv got EOF from 127.8.238.1:50903 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:49 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:49.922660 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:49.917946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:33123)
0811 10:12:49.918121 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:49.918131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:49.918190 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:49.918349 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:49.918350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:49.918526 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:49.918579 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.918580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.919227 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.919228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.919874 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:49.919877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.920210 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.920211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:49.920256 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.920786 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:49.920791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:49.922585 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
W20260811 10:12:49.922782 24854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:49.918025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:33123 (local address 127.8.238.62:44511)
0811 10:12:49.918080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:49.918097 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:49.918194 (+ 97us) server_negotiation.cc:407] Connection header received
0811 10:12:49.918236 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:49.918237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:49.918252 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:49.918319 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:49.918646 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.919151 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.919949 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:49.920141 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:49.920196 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:49.922681 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:33123: BlockingRecv error: recv got EOF from 127.8.238.2:33123 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:50.058744 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:50.053843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:38057)
0811 10:12:50.054035 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:50.054046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:50.054106 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:50.054254 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:50.054256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:50.054462 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:50.054520 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.054521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.055221 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.055223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.055855 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.055859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.056237 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.056238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.056307 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.056780 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:50.056785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:50.058628 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
W20260811 10:12:50.058777 24854 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:50.053931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:38057 (local address 127.8.238.62:44511)
0811 10:12:50.054007 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:50.054009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:50.054109 (+ 100us) server_negotiation.cc:407] Connection header received
0811 10:12:50.054168 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:50.054169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:50.054184 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:50.054228 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:50.054607 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.055128 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.055940 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.056150 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.056237 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.058707 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:38057: BlockingRecv error: recv got EOF from 127.8.238.3:38057 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:50.798789 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:50.793966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:39487)
0811 10:12:50.794141 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:50.794152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:50.794203 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:50.794342 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:50.794343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:50.794518 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:50.794577 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.794579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.795233 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.795234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.795831 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.795834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.796242 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.796243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.796294 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.796786 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:50.796791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:50.798728 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
W20260811 10:12:50.798944 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:50.794073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39487 (local address 127.8.238.62:44511)
0811 10:12:50.794194 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:50.794196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:50.794205 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:50.794251 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:50.794252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:50.794263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:50.794321 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:50.794644 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.795162 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.795907 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.796106 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.796157 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.798848 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:39487: BlockingRecv error: recv got EOF from 127.8.238.1:39487 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:50 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:50.928280 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:50.923152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:51731)
0811 10:12:50.923334 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:50.923346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:50.923422 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:50.923572 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:50.923573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:50.923792 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:50.923847 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.923849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.924606 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.924607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.925161 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:50.925163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.925557 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.925558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:50.925622 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.926109 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:50.926114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:50.928217 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
W20260811 10:12:50.928439 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:50.923239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:51731 (local address 127.8.238.62:44511)
0811 10:12:50.923306 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:50.923309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:50.923414 (+ 105us) server_negotiation.cc:407] Connection header received
0811 10:12:50.923483 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:50.923484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:50.923499 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:50.923559 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:50.923933 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.924511 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.925261 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:50.925478 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:50.925557 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:50.928357 (+ 2800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:51731: BlockingRecv error: recv got EOF from 127.8.238.2:51731 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:51.064347 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:51.059394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:41049)
0811 10:12:51.059610 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:51.059622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:51.059690 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:51.059848 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:51.059849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:51.060038 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:51.060094 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.060096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.060734 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.060735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.061387 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.061396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.061707 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.061707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.061747 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.062282 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:51.062287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:51.064252 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":139,"threads_started":1}
W20260811 10:12:51.064482 24860 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:51.059469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:41049 (local address 127.8.238.62:44511)
0811 10:12:51.059545 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:51.059548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:51.059690 (+ 142us) server_negotiation.cc:407] Connection header received
0811 10:12:51.059772 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:51.059773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:51.059786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:51.059828 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:51.060156 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.060655 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.061463 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.061648 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.061695 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.064409 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:41049: BlockingRecv error: recv got EOF from 127.8.238.3:41049 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:51.804253 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:51.799266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:34885)
0811 10:12:51.799419 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:51.799431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:51.799515 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:51.799722 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:51.799722 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:51.799972 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:51.800033 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.800036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.800851 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.800853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.801475 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.801478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.801844 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.801845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.801892 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.802376 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:51.802381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:51.804190 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260811 10:12:51.804445 24873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:51.799379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:34885 (local address 127.8.238.62:44511)
0811 10:12:51.799555 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:51.799558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:51.799568 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:12:51.799597 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:51.799598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:51.799613 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:51.799695 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:51.800140 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.800773 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.801561 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.801767 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.801830 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.804356 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:34885: BlockingRecv error: recv got EOF from 127.8.238.1:34885 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:51 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:51.933429 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:51.928768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:34525)
0811 10:12:51.928954 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:51.928984 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:51.929045 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:51.929180 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:51.929181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:51.929359 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:51.929415 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.929417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.930068 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.930069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.930591 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:51.930592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.930914 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.930915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:51.930956 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.931424 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:51.931429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:51.933321 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
W20260811 10:12:51.933478 24873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:51.928875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:34525 (local address 127.8.238.62:44511)
0811 10:12:51.928941 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:51.928943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:51.929032 (+ 89us) server_negotiation.cc:407] Connection header received
0811 10:12:51.929094 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:51.929095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:51.929106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:51.929156 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:51.929484 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.930000 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.930663 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:51.930849 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:51.930898 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:51.933410 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:34525: BlockingRecv error: recv got EOF from 127.8.238.2:34525 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:52.069962 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:52.064850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:51019)
0811 10:12:52.065059 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:52.065072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:52.065143 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:52.065295 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:52.065297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:52.065528 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:52.065587 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.065589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.066312 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.066313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.066879 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.066881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.067261 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.067263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.067338 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.067894 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:52.067899 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:52.069856 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":107,"threads_started":1}
W20260811 10:12:52.070024 24873 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:52.064959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:51019 (local address 127.8.238.62:44511)
0811 10:12:52.065043 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:52.065045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:52.065122 (+ 77us) server_negotiation.cc:407] Connection header received
0811 10:12:52.065199 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:52.065200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:52.065215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:52.065288 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:52.065677 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.066211 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.066976 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.067184 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.067263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.069956 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:51019: BlockingRecv error: recv got EOF from 127.8.238.3:51019 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:52.809651 24876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:52.804706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:59937)
0811 10:12:52.804861 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:52.804871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:52.804939 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:52.805099 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:52.805101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:52.805303 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:52.805356 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.805358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.806063 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.806064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.806640 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.806642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.807009 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.807010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.807062 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.807538 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:52.807543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:52.809547 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260811 10:12:52.809712 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:52.804771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:59937 (local address 127.8.238.62:44511)
0811 10:12:52.804907 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:52.804909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:52.804916 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:12:52.805005 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:52.805006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:52.805018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:52.805079 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:52.805453 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.805993 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.806709 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.806922 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.806984 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.809633 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:59937: BlockingRecv error: recv got EOF from 127.8.238.1:59937 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:52 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:52.938575 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:52.933903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:59841)
0811 10:12:52.934035 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:52.934045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:52.934112 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:52.934251 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:52.934252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:52.934424 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:52.934487 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.934488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.935114 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.935115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.935658 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:52.935659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.936046 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.936047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:52.936094 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.936629 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:52.936634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:52.938452 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
W20260811 10:12:52.938634 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:52.933982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:59841 (local address 127.8.238.62:44511)
0811 10:12:52.934057 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:52.934059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:52.934107 (+ 48us) server_negotiation.cc:407] Connection header received
0811 10:12:52.934171 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:52.934172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:52.934184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:52.934232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:52.934545 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.935049 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.935726 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:52.935978 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:52.936032 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:52.938585 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:59841: BlockingRecv error: recv got EOF from 127.8.238.2:59841 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:53.075577 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:53.070592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:52797)
0811 10:12:53.070765 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:53.070777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:53.070854 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:53.071037 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:53.071038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:53.071273 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:53.071347 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.071350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.072099 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.072100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.072637 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.072638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.073005 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.073006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.073066 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.073538 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:53.073543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:53.075500 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
W20260811 10:12:53.075640 24877 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:53.070703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:52797 (local address 127.8.238.62:44511)
0811 10:12:53.070765 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:53.070767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:53.070849 (+ 82us) server_negotiation.cc:407] Connection header received
0811 10:12:53.070911 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:53.070912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:53.070933 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:53.071003 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:53.071438 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.072028 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.072717 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.072913 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.072987 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.075588 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:52797: BlockingRecv error: recv got EOF from 127.8.238.3:52797 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:53.814807 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:53.810141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:58203)
0811 10:12:53.810314 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:53.810325 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:53.810389 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:53.810533 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:53.810534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:53.810710 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:53.810766 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.810768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.811477 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.811478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.812096 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.812105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.812439 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.812440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.812483 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.812950 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:53.812954 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:53.814744 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
W20260811 10:12:53.814934 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:53.810224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:58203 (local address 127.8.238.62:44511)
0811 10:12:53.810337 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:53.810339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:53.810375 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:12:53.810446 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:53.810447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:53.810458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:53.810516 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:53.810869 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.811397 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.812176 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.812370 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.812424 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.814844 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:58203: BlockingRecv error: recv got EOF from 127.8.238.1:58203 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:53 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:53.943909 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:53.939177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:48355)
0811 10:12:53.939388 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:53.939401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:53.939479 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:53.939612 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:53.939613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:53.939808 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:53.939888 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.939890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.940606 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.940607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.941149 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:53.941150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.941479 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.941480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:53.941520 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.941968 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:53.941973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:53.943833 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":121,"threads_started":1}
W20260811 10:12:53.944029 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:53.939209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:48355 (local address 127.8.238.62:44511)
0811 10:12:53.939287 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:53.939289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:53.939466 (+ 177us) server_negotiation.cc:407] Connection header received
0811 10:12:53.939540 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:53.939541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:53.939553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:53.939606 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:53.939972 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.940534 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.941226 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:53.941411 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:53.941465 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:53.943949 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:48355: BlockingRecv error: recv got EOF from 127.8.238.2:48355 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:54.081434 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:54.076185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:40509)
0811 10:12:54.076360 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:54.076372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:54.076447 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:54.076718 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:54.076720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:54.076915 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:54.076974 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.076976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.077699 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.077701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.078490 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.078493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.078836 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.078837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.078887 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.079451 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:54.079456 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:54.081366 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
W20260811 10:12:54.081543 24888 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:54.076244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:40509 (local address 127.8.238.62:44511)
0811 10:12:54.076324 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:54.076327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:54.076462 (+ 135us) server_negotiation.cc:407] Connection header received
0811 10:12:54.076600 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:54.076601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:54.076614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:54.076670 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:54.077051 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.077563 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.078559 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.078747 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.078805 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.081464 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:40509: BlockingRecv error: recv got EOF from 127.8.238.3:40509 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:54.824493 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:54.815488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:60561)
0811 10:12:54.815671 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:54.815685 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:54.815751 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:54.815958 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:54.815959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:54.816142 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:54.816202 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.816204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.816881 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.816881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.817499 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.817502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.818102 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.818103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.818150 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.822255 (+ 4105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:54.822261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:54.824389 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
W20260811 10:12:54.824572 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:54.815647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:60561 (local address 127.8.238.62:44511)
0811 10:12:54.815817 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:54.815819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:54.815827 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:54.815854 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:54.815854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:54.815866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:54.815934 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:54.816282 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.816809 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.817665 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.818016 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.818090 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.824493 (+ 6403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:60561: BlockingRecv error: recv got EOF from 127.8.238.1:60561 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:54 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:54.949337 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:54.944454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:54907)
0811 10:12:54.944615 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:54.944626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:54.944692 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:54.944838 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:54.944839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:54.945023 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:54.945079 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.945080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.945930 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.945932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.946485 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:54.946487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.946826 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.946827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:54.946886 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.947359 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:54.947364 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:54.949273 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
W20260811 10:12:54.949483 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:54.944504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:54907 (local address 127.8.238.62:44511)
0811 10:12:54.944587 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:54.944589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:54.944689 (+ 100us) server_negotiation.cc:407] Connection header received
0811 10:12:54.944738 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:54.944738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:54.944751 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:54.944800 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:54.945141 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.945809 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.946561 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:54.946758 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:54.946809 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:54.949391 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:54907: BlockingRecv error: recv got EOF from 127.8.238.2:54907 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:55.087229 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:55.081974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:56477)
0811 10:12:55.082265 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:55.082278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:55.082377 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:55.082575 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:55.082577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:55.082787 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:55.082843 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.082846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.083559 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.083561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.084187 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.084190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.084567 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.084569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.084632 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.085180 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:55.085186 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:55.087125 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":82,"threads_started":1}
W20260811 10:12:55.087293 24896 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:55.082153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:56477 (local address 127.8.238.62:44511)
0811 10:12:55.082241 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:55.082243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:55.082351 (+ 108us) server_negotiation.cc:407] Connection header received
0811 10:12:55.082462 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:55.082463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:55.082481 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:55.082565 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:55.082930 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.083458 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.084278 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.084482 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.084567 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.087222 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:56477: BlockingRecv error: recv got EOF from 127.8.238.3:56477 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:55.834100 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:55.825188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:35131)
0811 10:12:55.825456 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:55.825492 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:55.825585 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:55.825927 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:55.825928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:55.826278 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:55.826363 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.826371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.827647 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.827649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.828676 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.828680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.829436 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.829439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.829543 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.830590 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:55.830604 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:55.833985 (+ 3381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":173,"threads_started":1}
W20260811 10:12:55.834167 24902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:55.825404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:35131 (local address 127.8.238.62:44511)
0811 10:12:55.825643 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:55.825645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:55.825666 (+ 21us) server_negotiation.cc:407] Connection header received
0811 10:12:55.825710 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:55.825711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:55.825771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:55.825918 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:55.826470 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.827507 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.828789 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.829235 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.829418 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.834082 (+ 4664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:35131: BlockingRecv error: recv got EOF from 127.8.238.1:35131 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":137,"threads_started":1}
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:55 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:55.954584 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:55.949853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:44119)
0811 10:12:55.950044 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:55.950077 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:55.950152 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:55.950295 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:55.950296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:55.950483 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:55.950537 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.950539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.951193 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.951194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.951770 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:55.951772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.952109 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.952110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:55.952165 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.952631 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:55.952636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:55.954521 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
W20260811 10:12:55.954730 24902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:55.949934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:44119 (local address 127.8.238.62:44511)
0811 10:12:55.950038 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:55.950040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:55.950155 (+ 115us) server_negotiation.cc:407] Connection header received
0811 10:12:55.950232 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:55.950233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:55.950244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:55.950283 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:55.950603 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.951116 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.951835 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:55.952023 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:55.952071 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:55.954642 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:44119: BlockingRecv error: recv got EOF from 127.8.238.2:44119 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:56.093039 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:56.087785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:51095)
0811 10:12:56.087979 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:56.087993 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:56.088090 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:56.088260 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:56.088261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:56.088470 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:56.088526 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.088528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.089231 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.089232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.089785 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.089787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.090174 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.090175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.090238 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.090914 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:56.090920 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:56.092974 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":124,"threads_started":1}
W20260811 10:12:56.093214 24902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:56.087926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:51095 (local address 127.8.238.62:44511)
0811 10:12:56.087983 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:56.087986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:56.088070 (+ 84us) server_negotiation.cc:407] Connection header received
0811 10:12:56.088149 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:56.088150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:56.088167 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:56.088240 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:56.088616 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.089140 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.089879 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.090093 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.090173 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.093082 (+ 2909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:51095: BlockingRecv error: recv got EOF from 127.8.238.3:51095 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:56.839358 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:56.834606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:39187)
0811 10:12:56.834767 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:56.834779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:56.834861 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:56.835015 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:56.835016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:56.835203 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:56.835269 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.835271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.835951 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.835952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.836491 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.836493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.836817 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.836818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.836864 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.837354 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:56.837359 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:56.839254 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
W20260811 10:12:56.839417 24909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:56.834716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39187 (local address 127.8.238.62:44511)
0811 10:12:56.834858 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:56.834860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:56.834869 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:56.834916 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:56.834916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:56.834928 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:56.834996 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:56.835340 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.835884 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.836564 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.836753 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.836803 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.839337 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:39187: BlockingRecv error: recv got EOF from 127.8.238.1:39187 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:56 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:56.960394 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:56.955048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:50943)
0811 10:12:56.955276 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:56.955289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:56.955385 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:56.955573 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:56.955574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:56.955822 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:56.955876 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.955878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.956600 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.956602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.957167 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:56.957169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.957577 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.957578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:56.957648 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.958195 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:56.958200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:56.960290 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
W20260811 10:12:56.960455 24909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:56.955158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:50943 (local address 127.8.238.62:44511)
0811 10:12:56.955268 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:56.955271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:56.955365 (+ 94us) server_negotiation.cc:407] Connection header received
0811 10:12:56.955470 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:56.955471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:56.955486 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:56.955563 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:56.955965 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.956504 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.957267 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:56.957484 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:56.957574 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:56.960378 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:50943: BlockingRecv error: recv got EOF from 127.8.238.2:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:57.098345 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:57.093574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:33183)
0811 10:12:57.093733 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:57.093744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:57.093798 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:57.093957 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:57.093958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:57.094142 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:57.094200 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.094201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.094936 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.094938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.095507 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.095509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.095895 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.095896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.095940 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.096413 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:57.096417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:57.098278 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
W20260811 10:12:57.098497 24909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:57.093743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:33183 (local address 127.8.238.62:44511)
0811 10:12:57.093801 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:57.093803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:57.093831 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:12:57.093852 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:57.093853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:57.093866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:57.093943 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:57.094295 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.094835 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.095603 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.095819 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.095883 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.098413 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:33183: BlockingRecv error: recv got EOF from 127.8.238.3:33183 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:57.852031 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:57.839862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:50285)
0811 10:12:57.840031 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:57.840046 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:57.840123 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:57.840332 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:57.840334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:57.840615 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:57.840684 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.840687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.841433 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.841434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.841986 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.841988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.842402 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.842404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.842484 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.849574 (+ 7090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:57.849580 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:57.851910 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
W20260811 10:12:57.852095 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:57.839952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:50285 (local address 127.8.238.62:44511)
0811 10:12:57.840102 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:57.840104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:57.840113 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:12:57.840193 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:57.840194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:57.840209 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:57.840287 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:57.840788 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.841329 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.842100 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.842308 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.842371 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.852011 (+ 9640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:50285: BlockingRecv error: recv got EOF from 127.8.238.1:50285 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:57 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:57.966125 24920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:57.960912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:40041)
0811 10:12:57.961069 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:57.961080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:57.961152 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:57.961332 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:57.961335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:57.961543 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:57.961594 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.961596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.962360 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.962362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.962915 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:57.962917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.963303 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.963305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:57.963365 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.963918 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:57.963924 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:57.965990 (+ 2066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
W20260811 10:12:57.966190 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:57.960974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:40041 (local address 127.8.238.62:44511)
0811 10:12:57.961019 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:57.961021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:57.961141 (+ 120us) server_negotiation.cc:407] Connection header received
0811 10:12:57.961228 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:57.961229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:57.961241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:57.961328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:57.961689 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.962252 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.963018 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:57.963225 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:57.963305 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:57.966129 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:40041: BlockingRecv error: recv got EOF from 127.8.238.2:40041 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:58.103977 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:58.098840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:50555)
0811 10:12:58.099040 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:58.099051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:58.099113 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:58.099259 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:58.099260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:58.099455 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:58.099509 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.099511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.100401 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.100403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.101113 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.101116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.101473 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.101473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.101516 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.101969 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:58.101974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:58.103897 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
W20260811 10:12:58.104099 24919 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:58.098929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:50555 (local address 127.8.238.62:44511)
0811 10:12:58.099008 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:58.099010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:58.099112 (+ 102us) server_negotiation.cc:407] Connection header received
0811 10:12:58.099171 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:58.099172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:58.099187 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:58.099230 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:58.099575 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.100303 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.101204 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.101404 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.101458 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.103994 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:50555: BlockingRecv error: recv got EOF from 127.8.238.3:50555 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:58.857249 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:58.852643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:33811)
0811 10:12:58.852808 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:58.852823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:58.852892 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:58.853040 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:58.853041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:58.853228 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:58.853286 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.853288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.853938 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.853938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.854468 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.854470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.854797 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.854798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.854839 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.855323 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:58.855327 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:58.857170 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
W20260811 10:12:58.857437 24925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:58.852721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:33811 (local address 127.8.238.62:44511)
0811 10:12:58.852877 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:58.852879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:58.852887 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:12:58.852952 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:58.852953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:58.852965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:58.853016 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:58.853355 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.853872 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.854539 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.854733 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.854784 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.857315 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:33811: BlockingRecv error: recv got EOF from 127.8.238.1:33811 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:58 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:58.972108 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:58.967100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:35361)
0811 10:12:58.967319 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:58.967330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:58.967412 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:58.967552 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:58.967553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:58.967817 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:58.967879 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.967881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.968584 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.968585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.969162 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:58.969164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.969548 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.969549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:58.969615 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.970131 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:58.970136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:58.972033 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
W20260811 10:12:58.972242 24925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:58.967191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:35361 (local address 127.8.238.62:44511)
0811 10:12:58.967256 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:58.967258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:58.967395 (+ 137us) server_negotiation.cc:407] Connection header received
0811 10:12:58.967474 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:58.967474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:58.967489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:58.967538 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:58.967971 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.968494 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.969257 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:58.969467 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:58.969548 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:58.972135 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:35361: BlockingRecv error: recv got EOF from 127.8.238.2:35361 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:59.109558 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:59.104444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:36663)
0811 10:12:59.104581 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:59.104592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:59.104649 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:59.104776 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:59.104777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:59.104972 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:59.105024 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.105026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.105769 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.105771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.106319 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.106320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.106699 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.106701 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.106881 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.107511 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:59.107516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:59.109453 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
W20260811 10:12:59.109612 24925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:59.104528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:36663 (local address 127.8.238.62:44511)
0811 10:12:59.104634 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:59.104636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:59.104647 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:12:59.104707 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:59.104708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:59.104721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:59.104771 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:59.105085 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.105692 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.106403 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.106590 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.106647 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.109541 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:36663: BlockingRecv error: recv got EOF from 127.8.238.3:36663 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:59.865952 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:59.857807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:32933)
0811 10:12:59.857993 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:59.858010 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:59.858084 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:59.858226 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:59.858227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:59.858427 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:59.858483 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.858485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.859184 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.859185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.859729 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.859730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.860134 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.860135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.860186 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.863775 (+ 3589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:59.863781 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:59.865888 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":122,"threads_started":1}
W20260811 10:12:59.866091 24933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:59.857853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:32933 (local address 127.8.238.62:44511)
0811 10:12:59.858004 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:59.858006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:59.858065 (+ 59us) server_negotiation.cc:407] Connection header received
0811 10:12:59.858131 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:59.858132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:59.858142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:59.858203 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:59.858585 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.859112 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.859853 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.860064 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.860117 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.865997 (+ 5880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:32933: BlockingRecv error: recv got EOF from 127.8.238.1:32933 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:12:59 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:12:59.977519 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:12:59.972617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:58299)
0811 10:12:59.972809 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 10:12:59.972820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:12:59.972878 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:12:59.973002 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:12:59.973003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:12:59.973187 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:12:59.973241 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.973243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.973973 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.973974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.974525 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:12:59.974527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.974912 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.974913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:12:59.974976 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.975473 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:12:59.975477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:12:59.977438 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
W20260811 10:12:59.977659 24933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:12:59.972731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:58299 (local address 127.8.238.62:44511)
0811 10:12:59.972807 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 10:12:59.972809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:12:59.972862 (+ 53us) server_negotiation.cc:407] Connection header received
0811 10:12:59.972936 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:12:59.972937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:12:59.972948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:12:59.972997 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:12:59.973310 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.973850 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.974623 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:12:59.974831 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:12:59.974911 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:12:59.977545 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:58299: BlockingRecv error: recv got EOF from 127.8.238.2:58299 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:00.115458 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:00.110099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:55809)
0811 10:13:00.110255 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:00.110267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:00.110336 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:00.110484 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:00.110485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:00.110673 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:00.110726 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.110728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.111419 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.111420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.112295 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.112299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.112683 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.112684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.112743 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.113319 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:00.113324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:00.115371 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260811 10:13:00.115561 24933 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:00.110200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:55809 (local address 127.8.238.62:44511)
0811 10:13:00.110271 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:00.110277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:00.110334 (+ 57us) server_negotiation.cc:407] Connection header received
0811 10:13:00.110399 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:00.110399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:00.110413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:00.110459 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:00.110799 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.111307 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.112406 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.112609 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.112684 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.115463 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:55809: BlockingRecv error: recv got EOF from 127.8.238.3:55809 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:00.871856 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:00.866498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:44575)
0811 10:13:00.866712 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:00.866724 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:00.866793 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:00.866956 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:00.866958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:00.867161 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:00.867217 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.867219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.868096 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.868098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.868647 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.868648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.869054 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.869056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.869099 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.869615 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:00.869620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:00.871712 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":140,"threads_started":1}
W20260811 10:13:00.871913 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:00.866637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:44575 (local address 127.8.238.62:44511)
0811 10:13:00.866774 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:00.866776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:00.866785 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:13:00.866857 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:00.866857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:00.866869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:00.866934 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:00.867325 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.868007 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.868741 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.868979 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.869043 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.871851 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:44575: BlockingRecv error: recv got EOF from 127.8.238.1:44575 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:00 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:00.983304 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:00.978023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:57063)
0811 10:13:00.978192 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:00.978207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:00.978284 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:00.978525 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:00.978527 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:00.978728 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:00.978784 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.978786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.979462 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.979464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.980152 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:00.980154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.980520 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.980521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:00.980567 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.981343 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:00.981348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:00.983227 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
W20260811 10:13:00.983440 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:00.978081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:57063 (local address 127.8.238.62:44511)
0811 10:13:00.978160 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:00.978163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:00.978282 (+ 119us) server_negotiation.cc:407] Connection header received
0811 10:13:00.978409 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:00.978410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:00.978427 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:00.978496 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:00.978873 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.979383 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.980230 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:00.980445 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:00.980505 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:00.983352 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:57063: BlockingRecv error: recv got EOF from 127.8.238.2:57063 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.3@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:01.121579 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:01.116003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.3:59893)
0811 10:13:01.116245 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:01.116259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:01.116360 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:01.116550 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:01.116552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:01.116776 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:01.116833 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.116835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.117585 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.117586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.118285 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.118288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.118743 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.118744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.118803 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.119278 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:01.119284 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:01.121500 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":119,"threads_started":1}
W20260811 10:13:01.121713 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:01.116108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:59893 (local address 127.8.238.62:44511)
0811 10:13:01.116226 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:01.116229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:01.116337 (+ 108us) server_negotiation.cc:407] Connection header received
0811 10:13:01.116447 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:01.116448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:01.116471 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:01.116538 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:01.116930 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.117491 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.118410 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.118673 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.118727 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.121606 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.3:59893: BlockingRecv error: recv got EOF from 127.8.238.3:59893 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.1@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:01.878145 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:01.872485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.1:46281)
0811 10:13:01.872636 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:01.872650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:01.872732 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:01.872944 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:01.872945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:01.873155 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:01.873214 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.873216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.874004 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.874006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.874579 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.874582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.874942 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.874943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.874995 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.875542 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:01.875547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:01.878035 (+ 2488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
W20260811 10:13:01.878257 24949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:01.872581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:46281 (local address 127.8.238.62:44511)
0811 10:13:01.872780 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:01.872782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:01.872791 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:13:01.872815 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:01.872816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:01.872828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:01.872898 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:01.873300 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.873911 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.874654 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.874871 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.874929 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.878165 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.1:46281: BlockingRecv error: recv got EOF from 127.8.238.1:46281 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":80,"threads_started":1}
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 10:13:01 dist-test-slave-7f01 krb5kdc[23218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.238.2@KRBTEST.COM for oryx/127.8.238.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 10:13:01.989003 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:01.983938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44511 (local address 127.8.238.2:36731)
0811 10:13:01.984096 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:01.984108 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:01.984183 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:01.984341 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:01.984342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:01.984526 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:01.984582 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.984584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.985280 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.985281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.985855 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:01.985857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.986245 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.986247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:01.986313 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.986868 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:01.986881 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:01.988883 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.238.62:44511: Server oryx/127.8.238.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260811 10:13:01.989104 24949 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:01.984005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:36731 (local address 127.8.238.62:44511)
0811 10:13:01.984085 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:01.984087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:01.984198 (+ 111us) server_negotiation.cc:407] Connection header received
0811 10:13:01.984236 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:01.984236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:01.984250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:01.984320 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:01.984645 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.985178 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.985939 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:01.986154 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:01.986212 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:01.988999 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.238.2:36731: BlockingRecv error: recv got EOF from 127.8.238.2:36731 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 10:13:02.031977 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 24155
I20260811 10:13:02.048729 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 24406
I20260811 10:13:02.057935 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 24629
I20260811 10:13:02.067993 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 24049
2026-08-11T10:13:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17871 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T10:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:13:02Z Disabled control of system clock
I20260811 10:13:02.104550 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45889
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:42377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:45889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 10:13:02.193331 24956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:02.193536 24956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:02.193557 24956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:02.194826 24956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:13:02.194868 24956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:02.194881 24956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:02.194895 24956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:13:02.194906 24956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:13:02.196357 24956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:45889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:02.196609 24956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:02.196931 24956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:02.201845 24971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.201840 24968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.202060 24956 server_base.cc:1034] running on GCE node
W20260811 10:13:02.201840 24965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.202366 24956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:02.202625 24956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:02.203800 24956 hybrid_clock.cc:648] HybridClock initialized: now 1786443182203785 us; error 29 us; skew 500 ppm
I20260811 10:13:02.205091 24956 webserver.cc:460] Webserver started at http://127.8.238.62:44455/ using document root <none> and password file <none>
I20260811 10:13:02.205333 24956 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:02.205401 24956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:02.205580 24956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:02.206679 24956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "d1c3e7f1ace24334b41e67ca85457e65"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "7cbc120f1419bb3972d4d37a79a416e8"
server_key_iv: "77d247ca80049330df0fe381c65c8218"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.207070 24956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "d1c3e7f1ace24334b41e67ca85457e65"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "7cbc120f1419bb3972d4d37a79a416e8"
server_key_iv: "77d247ca80049330df0fe381c65c8218"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.208467 24956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:13:02.209199 24983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.209388 24956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:13:02.209513 24956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "d1c3e7f1ace24334b41e67ca85457e65"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "7cbc120f1419bb3972d4d37a79a416e8"
server_key_iv: "77d247ca80049330df0fe381c65c8218"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.209590 24956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:02.217862 24956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:02.218154 24956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:02.218299 24956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:02.222679 24956 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:45889
I20260811 10:13:02.222709 25078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:45889 every 8 connection(s)
I20260811 10:13:02.223033 24956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:13:02.223671 25080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:02.226311 25080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65: Bootstrap starting.
I20260811 10:13:02.226868 25080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:02.227118 25080 log.cc:824] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:02.227912 25080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65: No bootstrap required, opened a new log
I20260811 10:13:02.229457 25080 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } }
I20260811 10:13:02.229595 25080 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:02.229619 25080 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c3e7f1ace24334b41e67ca85457e65, State: Initialized, Role: FOLLOWER
I20260811 10:13:02.229753 25080 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } }
I20260811 10:13:02.229861 25080 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:13:02.229887 25080 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:13:02.229915 25080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:02.230422 25080 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } }
I20260811 10:13:02.230547 25080 leader_election.cc:302] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1c3e7f1ace24334b41e67ca85457e65; no voters:
I20260811 10:13:02.230610 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 24956
I20260811 10:13:02.230710 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:13:02.230722 25080 leader_election.cc:288] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:13:02.230916 25088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:02.230933 9144 external_mini_cluster.cc:1428] Setting key 569638253e33911358fef950538e3cc2
I20260811 10:13:02.230989 25080 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:13:02.231098 25088 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [term 1 LEADER]: Becoming Leader. State: Replica: d1c3e7f1ace24334b41e67ca85457e65, State: Running, Role: LEADER
I20260811 10:13:02.231212 25088 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } }
I20260811 10:13:02.231628 25085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1c3e7f1ace24334b41e67ca85457e65. Latest consensus state: current_term: 1 leader_uuid: "d1c3e7f1ace24334b41e67ca85457e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } } }
I20260811 10:13:02.231770 25085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:02.231848 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1c3e7f1ace24334b41e67ca85457e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c3e7f1ace24334b41e67ca85457e65" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45889 } } }
I20260811 10:13:02.231940 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:02.232116 25104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:13:02.232707 25104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:13:02.235483 25121 mvcc.cc:205] Tried to move back new op lower bound from 7317271274429018112 to 7317271274419429376. Current Snapshot: MvccSnapshot[applied={T|T < 7317271274429018112}]
I20260811 10:13:02.236024 25104 catalog_manager.cc:1347] Generated new cluster ID: 7ab1ce17ca074508b6c559365e36e2c2
I20260811 10:13:02.236080 25104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:13:02.237867 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.231457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.8.238.62:45889)
0811 10:13:02.231722 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.231730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.232222 (+ 492us) server_negotiation.cc:407] Connection header received
0811 10:13:02.232389 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.232398 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.232474 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.232614 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.232983 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.234565 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.235517 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.236201 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.236287 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.237181 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:02.237185 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:02.237186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:02.237370 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:02.237403 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:02.237414 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:02.237492 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:02.237577 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 10:13:02.237661 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260811 10:13:02.258509 25104 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:13:02.259199 25104 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:13:02.268838 25104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1c3e7f1ace24334b41e67ca85457e65: Generated new TSK 0
I20260811 10:13:02.269025 25104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:13:02.273734 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--builtin_ntp_servers=127.8.238.20:42377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 10:13:02.358676 25126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:02.358949 25126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:02.358987 25126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:02.360168 25126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:02.360240 25126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:02.360280 25126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:13:02.361631 25126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:02.361901 25126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:02.362236 25126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:02.368167 25141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.368289 25144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.368352 25126 server_base.cc:1034] running on GCE node
W20260811 10:13:02.368165 25138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.368772 25126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:02.369056 25126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:02.370206 25126 hybrid_clock.cc:648] HybridClock initialized: now 1786443182370180 us; error 41 us; skew 500 ppm
I20260811 10:13:02.371680 25126 webserver.cc:460] Webserver started at http://127.8.238.1:35235/ using document root <none> and password file <none>
I20260811 10:13:02.371965 25126 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:02.372036 25126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:02.372174 25126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:02.373314 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "35b6255c18854ebb9554b116ba3da835"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "0d777e004e6627f6460b61ac0cb0f26d"
server_key_iv: "3432d7653eb4d4b2a2590f8fa754e440"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.373623 25126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "35b6255c18854ebb9554b116ba3da835"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "0d777e004e6627f6460b61ac0cb0f26d"
server_key_iv: "3432d7653eb4d4b2a2590f8fa754e440"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.374936 25126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:13:02.375646 25155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.375877 25126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:13:02.375991 25126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "35b6255c18854ebb9554b116ba3da835"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "0d777e004e6627f6460b61ac0cb0f26d"
server_key_iv: "3432d7653eb4d4b2a2590f8fa754e440"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.376065 25126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:02.385041 25126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:02.385351 25126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:02.385500 25126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:02.385720 25126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:02.386060 25126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:02.386121 25126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.386163 25126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:02.386193 25126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.392441 25126 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:39897
I20260811 10:13:02.392562 25350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:39897 every 8 connection(s)
I20260811 10:13:02.392858 25126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:13:02.398149 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.393692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:33167 (local address 127.8.238.62:45889)
0811 10:13:02.393762 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.393765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.393942 (+ 177us) server_negotiation.cc:407] Connection header received
0811 10:13:02.394475 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.394476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.394489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.394546 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.395223 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.395786 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.396754 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.396958 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.397011 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.397831 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:02.397832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:02.397833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:02.397837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:02.397869 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:02.397873 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:02.397918 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:02.398009 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 10:13:02.398038 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:13:02.398288 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.393574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45889 (local address 127.8.238.1:33167)
0811 10:13:02.393790 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:02.393832 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:02.394382 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:02.394607 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:02.394610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:02.394618 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:02.395140 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.395144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.395895 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.395896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.396673 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.396676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.397038 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.397039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.397153 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.397690 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:02.397690 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:02.397718 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:02.397740 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:02.397931 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:02.397936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:02.397938 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:13:02.397989 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:13:02.398089 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260811 10:13:02.398838 25351 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45889
I20260811 10:13:02.399051 25351 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:02.399259 25351 heartbeater.cc:511] Master 127.8.238.62:45889 requested a full tablet report, sending...
I20260811 10:13:02.399843 25018 ts_manager.cc:194] Registered new tserver with Master: 35b6255c18854ebb9554b116ba3da835 (127.8.238.1:39897)
I20260811 10:13:02.399902 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 25126
I20260811 10:13:02.399994 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:13:02.400204 9144 external_mini_cluster.cc:1428] Setting key 275d542a644c0ddc6c214b86269ad847
I20260811 10:13:02.400769 25018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.1:33167
I20260811 10:13:02.401429 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--builtin_ntp_servers=127.8.238.20:42377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 10:13:02.486135 25354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:02.486336 25354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:02.486356 25354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:02.487493 25354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:02.487531 25354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:02.487555 25354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:13:02.488871 25354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:02.489060 25354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:02.489303 25354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:02.494333 25365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.494318 25363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.494560 25354 server_base.cc:1034] running on GCE node
W20260811 10:13:02.494304 25362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.494844 25354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:02.495092 25354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:02.496289 25354 hybrid_clock.cc:648] HybridClock initialized: now 1786443182496263 us; error 41 us; skew 500 ppm
I20260811 10:13:02.497704 25354 webserver.cc:460] Webserver started at http://127.8.238.2:46761/ using document root <none> and password file <none>
I20260811 10:13:02.497957 25354 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:02.498031 25354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:02.498229 25354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:02.499302 25354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "05cfa3cb15ac4062a52f76fc25edb1d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "24c2d3801e0543aca5ef6515f95f75dc"
server_key_iv: "77229dec977970ca18af97f4885210e6"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.499679 25354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "05cfa3cb15ac4062a52f76fc25edb1d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "24c2d3801e0543aca5ef6515f95f75dc"
server_key_iv: "77229dec977970ca18af97f4885210e6"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.501083 25354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:02.501816 25375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.502002 25354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:13:02.502113 25354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "05cfa3cb15ac4062a52f76fc25edb1d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "24c2d3801e0543aca5ef6515f95f75dc"
server_key_iv: "77229dec977970ca18af97f4885210e6"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.502188 25354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:02.519294 25354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:02.519603 25354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:02.519832 25354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:02.520073 25354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:02.520430 25354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:02.520488 25354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.520531 25354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:02.520560 25354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.526906 25354 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:41289
I20260811 10:13:02.526960 25571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:41289 every 8 connection(s)
I20260811 10:13:02.527288 25354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:13:02.532835 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.528038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:36885 (local address 127.8.238.62:45889)
0811 10:13:02.528089 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.528091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.528343 (+ 252us) server_negotiation.cc:407] Connection header received
0811 10:13:02.528936 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.528937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.528953 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.529023 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.529732 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.530244 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.531298 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.531495 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.531569 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.532517 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:02.532518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:02.532518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:02.532524 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:02.532539 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:02.532544 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:02.532613 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:02.532719 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 10:13:02.532751 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:13:02.533059 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.528015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45889 (local address 127.8.238.2:36885)
0811 10:13:02.528213 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:02.528247 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:02.528806 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:02.529119 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:02.529129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:02.529137 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:02.529634 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.529639 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.530353 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.530354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.531184 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.531188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.531593 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.531594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.531726 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.532342 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:02.532343 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:02.532372 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:02.532395 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:02.532629 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:02.532635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:02.532638 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:13:02.532721 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:13:02.532826 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260811 10:13:02.533510 25574 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45889
I20260811 10:13:02.533633 25574 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:02.533849 25574 heartbeater.cc:511] Master 127.8.238.62:45889 requested a full tablet report, sending...
I20260811 10:13:02.534260 25018 ts_manager.cc:194] Registered new tserver with Master: 05cfa3cb15ac4062a52f76fc25edb1d6 (127.8.238.2:41289)
I20260811 10:13:02.534641 25018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.2:36885
I20260811 10:13:02.537338 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 25354
I20260811 10:13:02.537427 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:13:02.537565 9144 external_mini_cluster.cc:1428] Setting key 0ee8f9aa342f69868fc54f3fd3755ff6
I20260811 10:13:02.538486 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--builtin_ntp_servers=127.8.238.20:42377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 10:13:02.623608 25578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:02.623885 25578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:02.623931 25578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:02.625186 25578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:02.625227 25578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:02.625252 25578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:13:02.626530 25578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:42377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:02.626710 25578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:02.626972 25578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:02.632246 25589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.632237 25587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.632298 25578 server_base.cc:1034] running on GCE node
W20260811 10:13:02.632241 25584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.632648 25578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:02.632936 25578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:02.634136 25578 hybrid_clock.cc:648] HybridClock initialized: now 1786443182634120 us; error 25 us; skew 500 ppm
I20260811 10:13:02.635442 25578 webserver.cc:460] Webserver started at http://127.8.238.3:40265/ using document root <none> and password file <none>
I20260811 10:13:02.635690 25578 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:02.635780 25578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:02.635949 25578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:02.637014 25578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "c53da0296ee74f55aef563366fe86194"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "2092eaa007426545aaeb680857374a6d"
server_key_iv: "8bd54fa3b52a826856e9ec065a981116"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.637419 25578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "c53da0296ee74f55aef563366fe86194"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "2092eaa007426545aaeb680857374a6d"
server_key_iv: "8bd54fa3b52a826856e9ec065a981116"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.638810 25578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:13:02.639621 25602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.639837 25578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:02.639945 25578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "c53da0296ee74f55aef563366fe86194"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "2092eaa007426545aaeb680857374a6d"
server_key_iv: "8bd54fa3b52a826856e9ec065a981116"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.640030 25578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:02.656351 25578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:02.656674 25578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:02.656833 25578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:02.657078 25578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:02.657423 25578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:02.657482 25578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.657526 25578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:02.657553 25578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.663548 25578 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43329
I20260811 10:13:02.663635 25792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43329 every 8 connection(s)
I20260811 10:13:02.663995 25578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:13:02.664592 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 25578
I20260811 10:13:02.664693 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:13:02.664911 9144 external_mini_cluster.cc:1428] Setting key 0ab8c08a2d684f6f80c142227d1d6047
I20260811 10:13:02.669853 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.664829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:46313 (local address 127.8.238.62:45889)
0811 10:13:02.664890 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.664894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.665148 (+ 254us) server_negotiation.cc:407] Connection header received
0811 10:13:02.665901 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.665902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.665915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.665960 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.666692 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.667232 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.668344 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.668568 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.668652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.669571 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:02.669572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:02.669573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:02.669578 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:02.669593 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:02.669597 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:02.669646 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:02.669734 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 10:13:02.669765 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:13:02.670055 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.664814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:45889 (local address 127.8.238.3:46313)
0811 10:13:02.665012 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:02.665054 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:02.665808 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:02.666029 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:02.666033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:02.666043 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:02.666587 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.666591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.667347 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.667348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.668227 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:02.668230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:02.668662 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:02.668663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:02.668811 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:02.669421 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:02.669421 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:02.669451 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:02.669474 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:02.669645 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:02.669651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:02.669654 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:13:02.669724 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 10:13:02.669828 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260811 10:13:02.670586 25794 heartbeater.cc:347] Connected to a master server at 127.8.238.62:45889
I20260811 10:13:02.670667 25794 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:02.670876 25794 heartbeater.cc:511] Master 127.8.238.62:45889 requested a full tablet report, sending...
I20260811 10:13:02.671273 25018 ts_manager.cc:194] Registered new tserver with Master: c53da0296ee74f55aef563366fe86194 (127.8.238.3:43329)
I20260811 10:13:02.671657 25018 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.3:46313
I20260811 10:13:02.676908 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 10:13:02.678905 25097 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:02.678385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.8.238.62:45889)
0811 10:13:02.678454 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.678456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.678560 (+ 104us) server_negotiation.cc:407] Connection header received
0811 10:13:02.678600 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.678601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.678611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.678668 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.678776 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33418: BlockingRecv error: recv got EOF from 127.0.0.1:33418 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:13:02.678994 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 25126
I20260811 10:13:02.690320 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 25354
I20260811 10:13:02.705893 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 25578
I20260811 10:13:02.720332 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 24956
2026-08-11T10:13:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (648 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T10:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:13:02Z Disabled control of system clock
I20260811 10:13:02.752148 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39887
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:13:02.834702 25816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:02.834914 25816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:02.834935 25816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:02.836068 25816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:13:02.836105 25816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:02.836118 25816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:02.836131 25816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:13:02.836144 25816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:13:02.837438 25816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:02.837633 25816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:02.837878 25816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:02.842834 25829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.842809 25833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:02.842746 25828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:02.842998 25816 server_base.cc:1034] running on GCE node
I20260811 10:13:02.843320 25816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:02.843660 25816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:02.844842 25816 hybrid_clock.cc:648] HybridClock initialized: now 1786443182844815 us; error 39 us; skew 500 ppm
I20260811 10:13:02.846187 25816 webserver.cc:460] Webserver started at http://127.8.238.62:40463/ using document root <none> and password file <none>
I20260811 10:13:02.846448 25816 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:02.846529 25816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:02.846683 25816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:02.847829 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "97f3ddbb745a4a81841654bc8eb2a2d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "733330cc793992af583c5d69b8c6ee20"
server_key_iv: "7913ec96eef0f7387698a59b56c0bcde"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.848201 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "97f3ddbb745a4a81841654bc8eb2a2d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "733330cc793992af583c5d69b8c6ee20"
server_key_iv: "7913ec96eef0f7387698a59b56c0bcde"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.849514 25816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:13:02.850237 25843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:02.850416 25816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:13:02.850538 25816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "97f3ddbb745a4a81841654bc8eb2a2d6"
format_stamp: "Formatted at 2026-08-11 10:13:02 on dist-test-slave-7f01"
server_key: "733330cc793992af583c5d69b8c6ee20"
server_key_iv: "7913ec96eef0f7387698a59b56c0bcde"
server_key_version: "encryptionkey@0"
I20260811 10:13:02.850615 25816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:02.854000 25816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:02.854270 25816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:02.854415 25816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:02.858634 25816 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39887
I20260811 10:13:02.858672 25931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39887 every 8 connection(s)
I20260811 10:13:02.859017 25816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:13:02.859684 25932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:02.862517 25932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6: Bootstrap starting.
I20260811 10:13:02.863080 25932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:02.863325 25932 log.cc:824] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:02.864023 25932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6: No bootstrap required, opened a new log
I20260811 10:13:02.865578 25932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } }
I20260811 10:13:02.865715 25932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:02.865737 25932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f3ddbb745a4a81841654bc8eb2a2d6, State: Initialized, Role: FOLLOWER
I20260811 10:13:02.865839 25932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } }
I20260811 10:13:02.865931 25932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:13:02.865981 25932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:13:02.866034 25932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:02.866452 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 25816
I20260811 10:13:02.866530 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:13:02.866631 25932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } }
I20260811 10:13:02.866766 9144 external_mini_cluster.cc:1428] Setting key 59191ae65313b8857216774392ecc40a
I20260811 10:13:02.866760 25932 leader_election.cc:302] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97f3ddbb745a4a81841654bc8eb2a2d6; no voters:
I20260811 10:13:02.866966 25932 leader_election.cc:288] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:13:02.866997 25936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:02.867117 25936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [term 1 LEADER]: Becoming Leader. State: Replica: 97f3ddbb745a4a81841654bc8eb2a2d6, State: Running, Role: LEADER
I20260811 10:13:02.867240 25932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:13:02.867201 25936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } }
I20260811 10:13:02.867915 25937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } } }
I20260811 10:13:02.868019 25937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:02.868201 25939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97f3ddbb745a4a81841654bc8eb2a2d6. Latest consensus state: current_term: 1 leader_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f3ddbb745a4a81841654bc8eb2a2d6" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39887 } } }
I20260811 10:13:02.868286 25939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:02.869038 25950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:13:02.869634 25950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:13:02.869633 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:02.867442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52584 (local address 127.8.238.62:39887)
0811 10:13:02.867663 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:02.867673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:02.867714 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:13:02.867874 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:02.867882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:02.867948 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:02.868528 (+ 580us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:02.869000 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:02.869003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:02.869003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:02.869007 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:02.869063 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:02.869079 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:02.869347 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:02.869381 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 10:13:02.869440 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260811 10:13:02.872812 25950 catalog_manager.cc:1347] Generated new cluster ID: 982800477bf54b1ca290ca758a78f996
I20260811 10:13:02.872877 25950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:13:02.901690 25950 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:13:02.901922 25950 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:13:02.906719 25950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97f3ddbb745a4a81841654bc8eb2a2d6: Generated new TSK 0
I20260811 10:13:02.906945 25950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:13:02.918216 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--builtin_ntp_servers=127.8.238.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:13:03.013423 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:03.013614 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:03.013636 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:03.014791 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:03.014829 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:03.014864 25976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:13:03.016239 25976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:03.016425 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:03.016661 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:13:03.021528 25976 server_base.cc:1034] running on GCE node
W20260811 10:13:03.021595 25997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:03.021519 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:03.021550 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:03.021994 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:03.022225 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:03.023377 25976 hybrid_clock.cc:648] HybridClock initialized: now 1786443183023363 us; error 29 us; skew 500 ppm
I20260811 10:13:03.024679 25976 webserver.cc:460] Webserver started at http://127.8.238.1:33455/ using document root <none> and password file <none>
I20260811 10:13:03.024927 25976 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:03.025002 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:03.025159 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:03.026284 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "09a4b379be254ad19e946a85901de567"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "5bb41b961ad6775c777d3a80e3cb4562"
server_key_iv: "683afd96257d85ec652c627deeb5c0b0"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.026667 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "09a4b379be254ad19e946a85901de567"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "5bb41b961ad6775c777d3a80e3cb4562"
server_key_iv: "683afd96257d85ec652c627deeb5c0b0"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.028026 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:13:03.028743 26006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.028923 25976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.029016 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "09a4b379be254ad19e946a85901de567"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "5bb41b961ad6775c777d3a80e3cb4562"
server_key_iv: "683afd96257d85ec652c627deeb5c0b0"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.029096 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:03.032505 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:03.032769 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:03.032912 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:03.033131 25976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:03.033443 25976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:03.033497 25976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.033535 25976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:03.033561 25976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.039793 25976 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:45325
I20260811 10:13:03.039821 26205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:45325 every 8 connection(s)
I20260811 10:13:03.040134 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:13:03.043196 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.040804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39557 (local address 127.8.238.62:39887)
0811 10:13:03.040851 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:03.040853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:03.041119 (+ 266us) server_negotiation.cc:407] Connection header received
0811 10:13:03.041638 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:03.041639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:03.041650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:03.041727 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:03.042917 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:03.042918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:03.042918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:03.042924 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:03.042943 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:03.042950 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:03.043024 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:03.043102 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 10:13:03.043121 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:13:03.043397 26211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.040797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39887 (local address 127.8.238.1:39557)
0811 10:13:03.041000 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:03.041032 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:03.041519 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:03.041802 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:03.041806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:03.041815 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:03.042713 (+ 898us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:03.042713 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:03.042740 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:03.042767 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:03.043020 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:03.043028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:03.043031 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:13:03.043114 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 10:13:03.043174 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260811 10:13:03.043934 26206 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39887
I20260811 10:13:03.044029 26206 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:03.044140 26206 heartbeater.cc:511] Master 127.8.238.62:39887 requested a full tablet report, sending...
I20260811 10:13:03.044653 25876 ts_manager.cc:194] Registered new tserver with Master: 09a4b379be254ad19e946a85901de567 (127.8.238.1:45325)
I20260811 10:13:03.045146 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 25976
I20260811 10:13:03.045245 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:13:03.045449 9144 external_mini_cluster.cc:1428] Setting key 719e31bc30fc5d765d5710aac9e16f48
I20260811 10:13:03.046289 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--builtin_ntp_servers=127.8.238.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:13:03.130142 26212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:03.130358 26212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:03.130381 26212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:03.131671 26212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:03.131726 26212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:03.131804 26212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:13:03.133379 26212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:03.133643 26212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:03.133956 26212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:03.138981 26224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:03.139006 26223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:03.139038 26212 server_base.cc:1034] running on GCE node
W20260811 10:13:03.139006 26232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:03.139421 26212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:03.139618 26212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:03.140841 26212 hybrid_clock.cc:648] HybridClock initialized: now 1786443183140821 us; error 50 us; skew 500 ppm
I20260811 10:13:03.142263 26212 webserver.cc:460] Webserver started at http://127.8.238.2:39367/ using document root <none> and password file <none>
I20260811 10:13:03.142509 26212 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:03.142555 26212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:03.142730 26212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:03.143927 26212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "b4388e674c69484b8a22eeb8d26150bf"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "598ee4ca8da060ea819410978e94b48b"
server_key_iv: "975e0caa610e9c0070d5dd9470b39a44"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.144284 26212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "b4388e674c69484b8a22eeb8d26150bf"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "598ee4ca8da060ea819410978e94b48b"
server_key_iv: "975e0caa610e9c0070d5dd9470b39a44"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.145637 26212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:13:03.146368 26239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.146540 26212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:13:03.146642 26212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "b4388e674c69484b8a22eeb8d26150bf"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "598ee4ca8da060ea819410978e94b48b"
server_key_iv: "975e0caa610e9c0070d5dd9470b39a44"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.146726 26212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:03.150110 26212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:03.150367 26212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:03.150513 26212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:03.150718 26212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:03.151046 26212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:03.151103 26212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.151144 26212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:03.151171 26212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.157261 26212 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:39681
I20260811 10:13:03.157322 26429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:39681 every 8 connection(s)
I20260811 10:13:03.157600 26212 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:13:03.160472 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.158275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:41101 (local address 127.8.238.62:39887)
0811 10:13:03.158334 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:03.158336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:03.158553 (+ 217us) server_negotiation.cc:407] Connection header received
0811 10:13:03.158975 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:03.158976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:03.158982 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:03.159034 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:03.160229 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:03.160231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:03.160231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:03.160235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:03.160252 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:03.160257 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:03.160311 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:03.160399 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 10:13:03.160412 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:13:03.160650 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.158220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39887 (local address 127.8.238.2:41101)
0811 10:13:03.158430 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:03.158461 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:03.158890 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:03.159082 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:03.159085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:03.159093 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:03.160063 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:03.160063 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:03.160091 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:03.160119 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:03.160323 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:03.160328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:03.160330 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:13:03.160389 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:13:03.160437 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":67,"threads_started":1}
I20260811 10:13:03.161127 26430 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39887
I20260811 10:13:03.161221 26430 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:03.161339 26430 heartbeater.cc:511] Master 127.8.238.62:39887 requested a full tablet report, sending...
I20260811 10:13:03.161664 25876 ts_manager.cc:194] Registered new tserver with Master: b4388e674c69484b8a22eeb8d26150bf (127.8.238.2:39681)
I20260811 10:13:03.162186 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 26212
I20260811 10:13:03.162279 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:13:03.162429 9144 external_mini_cluster.cc:1428] Setting key 73a4cee0a78a4ac0abbe3abda4be9ea1
I20260811 10:13:03.163236 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--builtin_ntp_servers=127.8.238.20:40087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 10:13:03.248832 26434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:03.249032 26434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:03.249109 26434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:03.250379 26434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:03.250420 26434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:03.250447 26434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:13:03.251721 26434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:03.251962 26434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:03.252210 26434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:03.257308 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:03.257324 26443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:03.257511 26434 server_base.cc:1034] running on GCE node
W20260811 10:13:03.257542 26442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:03.257790 26434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:03.258077 26434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:03.259219 26434 hybrid_clock.cc:648] HybridClock initialized: now 1786443183259209 us; error 28 us; skew 500 ppm
I20260811 10:13:03.260630 26434 webserver.cc:460] Webserver started at http://127.8.238.3:33187/ using document root <none> and password file <none>
I20260811 10:13:03.260870 26434 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:03.260917 26434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:03.261070 26434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:03.262138 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "f6d67a6ad908450c8d4000e158aba853"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "b7ccfba1508e878ee4e18412f59a3ad1"
server_key_iv: "f71b0b8860ededbae032c8b20039791b"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.262512 26434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "f6d67a6ad908450c8d4000e158aba853"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "b7ccfba1508e878ee4e18412f59a3ad1"
server_key_iv: "f71b0b8860ededbae032c8b20039791b"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.263846 26434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:13:03.264601 26457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.264792 26434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:13:03.264889 26434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "f6d67a6ad908450c8d4000e158aba853"
format_stamp: "Formatted at 2026-08-11 10:13:03 on dist-test-slave-7f01"
server_key: "b7ccfba1508e878ee4e18412f59a3ad1"
server_key_iv: "f71b0b8860ededbae032c8b20039791b"
server_key_version: "encryptionkey@0"
I20260811 10:13:03.264971 26434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:03.268440 26434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:03.268712 26434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:03.268848 26434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:03.269083 26434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:03.269392 26434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:03.269424 26434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.269474 26434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:03.269501 26434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:03.275276 26434 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:36959
I20260811 10:13:03.275343 26648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:36959 every 8 connection(s)
I20260811 10:13:03.275638 26434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:13:03.278574 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.276409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:41601 (local address 127.8.238.62:39887)
0811 10:13:03.276459 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:03.276461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:03.276614 (+ 153us) server_negotiation.cc:407] Connection header received
0811 10:13:03.277091 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:03.277093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:03.277101 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:03.277164 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:03.278346 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:03.278348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:03.278348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:13:03.278353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:03.278370 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:13:03.278382 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:03.278446 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:03.278496 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 10:13:03.278516 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:13:03.278724 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:03.276326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39887 (local address 127.8.238.3:41601)
0811 10:13:03.276509 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:03.276546 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:03.276976 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:03.277211 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:03.277214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:03.277222 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:03.278179 (+ 957us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:13:03.278179 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:03.278206 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:13:03.278234 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:03.278445 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:03.278448 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:03.278450 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:13:03.278500 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 10:13:03.278545 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260811 10:13:03.278904 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 26434
I20260811 10:13:03.278988 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:13:03.279176 9144 external_mini_cluster.cc:1428] Setting key 9de6d18b7aa4ada4cecbae38dfb010fb
I20260811 10:13:03.279290 26650 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39887
I20260811 10:13:03.279388 26650 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:03.279543 26650 heartbeater.cc:511] Master 127.8.238.62:39887 requested a full tablet report, sending...
I20260811 10:13:03.280072 25875 ts_manager.cc:194] Registered new tserver with Master: f6d67a6ad908450c8d4000e158aba853 (127.8.238.3:36959)
I20260811 10:13:03.291069 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 10:13:03.292958 25942 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:13:03.292507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.8.238.62:39887)
0811 10:13:03.292561 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:03.292562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:03.292685 (+ 123us) server_negotiation.cc:407] Connection header received
0811 10:13:03.292738 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:03.292739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:03.292744 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:03.292796 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:03.292905 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52594: BlockingRecv error: recv got EOF from 127.0.0.1:52594 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 10:13:03.293030 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 25976
I20260811 10:13:03.314005 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 26212
I20260811 10:13:03.327436 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 26434
I20260811 10:13:03.339622 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 25816
2026-08-11T10:13:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (624 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:13:03 dist-test-slave-7f01 krb5kdc[26667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:13:03 dist-test-slave-7f01 krb5kdc[26667](info): set up 2 sockets
Aug 11 10:13:03 dist-test-slave-7f01 krb5kdc[26667](info): commencing operation
krb5kdc: starting...
W20260811 10:13:04.624176 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:13:04 dist-test-slave-7f01 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:13:04.748704 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35215
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:04.830647 26697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:04.830855 26697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:04.830878 26697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:04.831997 26697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:13:04.832033 26697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:04.832047 26697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:04.832059 26697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:13:04.832072 26697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:13:04.833498 26697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:04.833689 26697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:04.833856 26697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:04.838708 26705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:04.838682 26709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:04.838917 26697 server_base.cc:1034] running on GCE node
W20260811 10:13:04.838694 26707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:04.839239 26697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:04.839538 26697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:04.840703 26697 hybrid_clock.cc:648] HybridClock initialized: now 1786443184840687 us; error 27 us; skew 500 ppm
Aug 11 10:13:04 dist-test-slave-7f01 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443184, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:04.843436 26697 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:13:04.843931 26697 webserver.cc:460] Webserver started at http://127.8.238.62:40267/ using document root <none> and password file <none>
I20260811 10:13:04.844158 26697 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:04.844228 26697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:04.844389 26697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:04.845451 26697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "f9a61816fdd14b44995dbf3e47e25d3e"
format_stamp: "Formatted at 2026-08-11 10:13:04 on dist-test-slave-7f01"
server_key: "a0987764c57ef5cb4d0fb09a4d6f35fb"
server_key_iv: "a03226ac14f3dc53ce1240d4a764d89a"
server_key_version: "encryptionkey@0"
I20260811 10:13:04.845830 26697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "f9a61816fdd14b44995dbf3e47e25d3e"
format_stamp: "Formatted at 2026-08-11 10:13:04 on dist-test-slave-7f01"
server_key: "a0987764c57ef5cb4d0fb09a4d6f35fb"
server_key_iv: "a03226ac14f3dc53ce1240d4a764d89a"
server_key_version: "encryptionkey@0"
I20260811 10:13:04.847186 26697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:13:04.847965 26721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:04.848142 26697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:04.848256 26697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "f9a61816fdd14b44995dbf3e47e25d3e"
format_stamp: "Formatted at 2026-08-11 10:13:04 on dist-test-slave-7f01"
server_key: "a0987764c57ef5cb4d0fb09a4d6f35fb"
server_key_iv: "a03226ac14f3dc53ce1240d4a764d89a"
server_key_version: "encryptionkey@0"
I20260811 10:13:04.848332 26697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:04.864275 26697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:04.869678 26697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:04.869871 26697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:04.874385 26697 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35215
I20260811 10:13:04.874445 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35215 every 8 connection(s)
I20260811 10:13:04.874826 26697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:13:04.875874 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:04.878414 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: Bootstrap starting.
I20260811 10:13:04.879063 26802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:04.879401 26802 log.cc:824] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:04.880203 26802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: No bootstrap required, opened a new log
I20260811 10:13:04.882131 26802 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } }
I20260811 10:13:04.882354 26802 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:04.882411 26802 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9a61816fdd14b44995dbf3e47e25d3e, State: Initialized, Role: FOLLOWER
I20260811 10:13:04.882589 26802 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } }
I20260811 10:13:04.882678 26802 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:13:04.882732 26802 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:13:04.882776 26802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:04.883592 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 26697
I20260811 10:13:04.883711 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:13:04.883792 26802 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } }
I20260811 10:13:04.883915 26802 leader_election.cc:302] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9a61816fdd14b44995dbf3e47e25d3e; no voters:
I20260811 10:13:04.883966 9144 external_mini_cluster.cc:1428] Setting key 8ab25d4eef54dfe167259ab067451fd1
I20260811 10:13:04.884090 26802 leader_election.cc:288] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:13:04.884124 26806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:04.884320 26806 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [term 1 LEADER]: Becoming Leader. State: Replica: f9a61816fdd14b44995dbf3e47e25d3e, State: Running, Role: LEADER
I20260811 10:13:04.884382 26802 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:13:04.884474 26806 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } }
I20260811 10:13:04.884809 26806 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9a61816fdd14b44995dbf3e47e25d3e. Latest consensus state: current_term: 1 leader_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } } }
I20260811 10:13:04.884897 26806 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:04.884857 26809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9a61816fdd14b44995dbf3e47e25d3e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35215 } } }
I20260811 10:13:04.885089 26809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e [sys.catalog]: This master's current role is: LEADER
W20260811 10:13:04.886548 26838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:13:04.886596 26838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:13:04.886711 26817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:13:04.886920 26817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:13:04.888482 26817 catalog_manager.cc:1347] Generated new cluster ID: fa89aa39d8cc47e28535b7f813067e3c
I20260811 10:13:04.888540 26817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:13:04 dist-test-slave-7f01 krb5kdc[26667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:04.895676 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:04.884655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49120 (local address 127.8.238.62:35215)
0811 10:13:04.884932 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:04.884942 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:04.884980 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:13:04.885120 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:04.885129 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:04.885197 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:04.885362 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:04.886199 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:04.887702 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:04.888699 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:04.889138 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:04.889269 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:04.891801 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:04.891805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:04.891805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:04.891819 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:04.894038 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:04.894431 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:04.894434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:04.894435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:04.894549 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:04.894782 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:04.894783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:04.894783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:04.895032 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:04.895207 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:04.895318 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 10:13:04.895467 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260811 10:13:04.907459 26817 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:13:04.908066 26817 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:13:04.914938 26817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: Generated new TSK 0
I20260811 10:13:04.915138 26817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:13:04.950897 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:05.037755 26846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:05.037961 26846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:05.037983 26846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:05.039145 26846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:05.039187 26846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:05.039242 26846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:13:05.040781 26846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:05.041044 26846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:05.041276 26846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:05.046414 26859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:05.046388 26857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:05.046447 26846 server_base.cc:1034] running on GCE node
W20260811 10:13:05.046379 26853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:05.046821 26846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:05.047070 26846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:05.048238 26846 hybrid_clock.cc:648] HybridClock initialized: now 1786443185048226 us; error 30 us; skew 500 ppm
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:05.050805 26846 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:13:05.051261 26846 webserver.cc:460] Webserver started at http://127.8.238.1:45623/ using document root <none> and password file <none>
I20260811 10:13:05.051504 26846 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:05.051575 26846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:05.051728 26846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:05.052836 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "ec6fb3342c3846a49860af143fddc2f9"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "b20c2bdc59c30eced8f71166f2833464"
server_key_iv: "e8633463fec03c7985a7b20e1cf6b05d"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.053210 26846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "ec6fb3342c3846a49860af143fddc2f9"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "b20c2bdc59c30eced8f71166f2833464"
server_key_iv: "e8633463fec03c7985a7b20e1cf6b05d"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.054572 26846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:13:05.055294 26867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.055476 26846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:13:05.055588 26846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "ec6fb3342c3846a49860af143fddc2f9"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "b20c2bdc59c30eced8f71166f2833464"
server_key_iv: "e8633463fec03c7985a7b20e1cf6b05d"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.055663 26846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:05.073733 26846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:05.075855 26846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:05.076027 26846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:05.076267 26846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:05.076602 26846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:05.076659 26846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.076695 26846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:05.076723 26846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.082978 26846 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:42021
I20260811 10:13:05.083102 27047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:42021 every 8 connection(s)
I20260811 10:13:05.083395 26846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:13:05.086705 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 26846
I20260811 10:13:05.086844 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:13:05.087051 9144 external_mini_cluster.cc:1428] Setting key 982601f673e924e4f2dd3b4cd8a91e4e
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:05.094388 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.084199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:42721 (local address 127.8.238.62:35215)
0811 10:13:05.084272 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.084275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.084584 (+ 309us) server_negotiation.cc:407] Connection header received
0811 10:13:05.085119 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.085121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.085155 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.085227 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:05.086420 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.087289 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.088532 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.088859 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.088939 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.091344 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:05.091346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:05.091346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:05.091354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:05.092807 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.093244 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.093246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.093246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.093379 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.093648 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.093649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.093650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.093811 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:05.093896 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.094183 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.094294 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:13:05.094550 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.084171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35215 (local address 127.8.238.1:42721)
0811 10:13:05.084455 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.084499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.085015 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.085283 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.085287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.085958 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:05.086209 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.086215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.087418 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.087419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.088432 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.088435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.088949 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.088950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.089059 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.089556 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:05.089584 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:05.091244 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:05.092895 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.092903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.092905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.093148 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.093455 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.093456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.093457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.093577 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.093886 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:05.093894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:05.094043 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.094156 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.094316 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":150,"threads_started":1}
I20260811 10:13:05.095131 27050 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35215
I20260811 10:13:05.095228 27050 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:05.095425 27050 heartbeater.cc:511] Master 127.8.238.62:35215 requested a full tablet report, sending...
I20260811 10:13:05.096007 26749 ts_manager.cc:194] Registered new tserver with Master: ec6fb3342c3846a49860af143fddc2f9 (127.8.238.1:42021)
I20260811 10:13:05.096633 26749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:42721
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:13:05.115989 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:05.202003 27061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:05.202214 27061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:05.202239 27061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:05.203431 27061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:05.203474 27061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:05.203501 27061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:13:05.205052 27061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:05.205300 27061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:05.205547 27061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:05.210750 27072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:05.210809 27061 server_base.cc:1034] running on GCE node
W20260811 10:13:05.210772 27068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:05.210752 27066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:05.211129 27061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:05.211338 27061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:05.212522 27061 hybrid_clock.cc:648] HybridClock initialized: now 1786443185212499 us; error 33 us; skew 500 ppm
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:05.215238 27061 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:13:05.215713 27061 webserver.cc:460] Webserver started at http://127.8.238.2:37281/ using document root <none> and password file <none>
I20260811 10:13:05.215989 27061 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:05.216037 27061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:05.216192 27061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:05.217229 27061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "a9d65f256d0a4f48988b731cfad28587"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "c0049399f047c36e4fe9f79c799efdea"
server_key_iv: "654e46b6d4e599c07d73549877f61d75"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.217603 27061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "a9d65f256d0a4f48988b731cfad28587"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "c0049399f047c36e4fe9f79c799efdea"
server_key_iv: "654e46b6d4e599c07d73549877f61d75"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.218906 27061 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:13:05.219656 27084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.219892 27061 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:05.219998 27061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "a9d65f256d0a4f48988b731cfad28587"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "c0049399f047c36e4fe9f79c799efdea"
server_key_iv: "654e46b6d4e599c07d73549877f61d75"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.220055 27061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:05.241945 27061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:05.244055 27061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:05.244230 27061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:05.244508 27061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:05.244930 27061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:05.244987 27061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.245021 27061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:05.245051 27061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.251024 27061 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:44263
I20260811 10:13:05.251088 27276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:44263 every 8 connection(s)
I20260811 10:13:05.251466 27061 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:05.260478 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.252521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43795 (local address 127.8.238.62:35215)
0811 10:13:05.252609 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.252612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.252626 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:13:05.253130 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.253131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.253142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.253192 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:05.254098 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.254611 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.255551 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.255799 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.255869 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.257821 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:05.257823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:05.257824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:05.257831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:05.259241 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.259639 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.259640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.259641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.259695 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.259951 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.259952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.259952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.260062 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:05.260116 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.260345 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.260413 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:13:05.260651 27280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.252190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35215 (local address 127.8.238.2:43795)
0811 10:13:05.252401 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.252448 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.253035 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.253243 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.253247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.253820 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:05.254020 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.254025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.254692 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.254693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.255473 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.255475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.255878 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.255879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.255991 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.256403 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:05.256411 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:05.257737 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:05.259308 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.259313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.259315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.259573 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.259792 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.259793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.259793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.259892 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.260119 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:05.260123 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:05.260232 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.260312 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.260429 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":130,"threads_started":1}
I20260811 10:13:05.261196 27278 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35215
I20260811 10:13:05.261304 27278 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:05.261330 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 27061
I20260811 10:13:05.261426 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:13:05.261509 27278 heartbeater.cc:511] Master 127.8.238.62:35215 requested a full tablet report, sending...
I20260811 10:13:05.261655 9144 external_mini_cluster.cc:1428] Setting key ea2eb9b3da6de94465c3ddb653b4d7c0
I20260811 10:13:05.261904 26749 ts_manager.cc:194] Registered new tserver with Master: a9d65f256d0a4f48988b731cfad28587 (127.8.238.2:44263)
I20260811 10:13:05.262427 26749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:43795
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:13:05.287830 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:05.369831 27284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:05.370064 27284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:05.370091 27284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:05.371263 27284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:05.371302 27284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:05.371328 27284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:13:05.372776 27284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:05.372948 27284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:05.373124 27284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:05.378114 27292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:05.378129 27293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:05.378093 27295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:05.378369 27284 server_base.cc:1034] running on GCE node
I20260811 10:13:05.378584 27284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:05.378808 27284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:05.379977 27284 hybrid_clock.cc:648] HybridClock initialized: now 1786443185379953 us; error 37 us; skew 500 ppm
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:05.382417 27284 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:13:05.382851 27284 webserver.cc:460] Webserver started at http://127.8.238.3:44457/ using document root <none> and password file <none>
I20260811 10:13:05.383080 27284 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:05.383148 27284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:05.383281 27284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:13:05.384471 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "d419160b6f2744d48f3a6a510754e64a"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "7520d27bea40b2be8a1c627aa804c9d7"
server_key_iv: "975f1c0fec2a34589bc0c5e0bdc53c38"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.384863 27284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "d419160b6f2744d48f3a6a510754e64a"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "7520d27bea40b2be8a1c627aa804c9d7"
server_key_iv: "975f1c0fec2a34589bc0c5e0bdc53c38"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.386404 27284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:13:05.387233 27305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.387439 27284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:05.387548 27284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "d419160b6f2744d48f3a6a510754e64a"
format_stamp: "Formatted at 2026-08-11 10:13:05 on dist-test-slave-7f01"
server_key: "7520d27bea40b2be8a1c627aa804c9d7"
server_key_iv: "975f1c0fec2a34589bc0c5e0bdc53c38"
server_key_version: "encryptionkey@0"
I20260811 10:13:05.387632 27284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:05.423218 27284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:05.425297 27284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:05.425474 27284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:05.425726 27284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:05.426052 27284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:05.426108 27284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.426151 27284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:05.426185 27284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:05.432696 27284 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:41821
I20260811 10:13:05.432752 27485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:41821 every 8 connection(s)
I20260811 10:13:05.433071 27284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[26667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:05.441799 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.433811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:46185 (local address 127.8.238.62:35215)
0811 10:13:05.433882 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.433883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.434062 (+ 179us) server_negotiation.cc:407] Connection header received
0811 10:13:05.434563 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.434564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.434575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.434633 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:05.435545 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.436139 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.437089 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.437275 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.437327 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.439342 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:05.439344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:05.439344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:05.439351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:05.440634 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.441016 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.441017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.441018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.441071 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.441288 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.441289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.441289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.441393 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:05.441444 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.441673 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.441735 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:13:05.441958 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.433781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35215 (local address 127.8.238.3:46185)
0811 10:13:05.433934 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.433977 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.434480 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.434687 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.434690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.435257 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:05.435447 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.435452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.436222 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.436223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.437011 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.437013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.437349 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.437350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.437447 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.437889 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:05.437897 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:05.439255 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:05.440706 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.440713 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.440715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.440946 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.441132 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:05.441132 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:05.441133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:05.441232 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:05.441448 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:05.441453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:05.441562 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.441642 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.441755 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260811 10:13:05.442483 27486 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35215
I20260811 10:13:05.442577 27486 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:05.442718 27486 heartbeater.cc:511] Master 127.8.238.62:35215 requested a full tablet report, sending...
I20260811 10:13:05.442991 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 27284
I20260811 10:13:05.443094 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:13:05.443190 26749 ts_manager.cc:194] Registered new tserver with Master: d419160b6f2744d48f3a6a510754e64a (127.8.238.3:41821)
I20260811 10:13:05.443312 9144 external_mini_cluster.cc:1428] Setting key 5f0af851c06a9894a0364850822ee3fd
I20260811 10:13:05.443611 26749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:46185
I20260811 10:13:05.444324 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:13:05.451469 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.445639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.8.238.62:35215)
0811 10:13:05.445707 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.445709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.445783 (+ 74us) server_negotiation.cc:407] Connection header received
0811 10:13:05.445857 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.445858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.445868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.445910 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:05.446469 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.446936 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.447660 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.447941 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.448048 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.449135 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:05.449136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:05.449137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:05.449143 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:05.450316 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.450656 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.450658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.450658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.450713 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:05.450934 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:05.450935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:05.450936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:05.451088 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:05.451198 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.451298 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.451375 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:13:05.452934 26749 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:13:05.454558 26749 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:13:05.461303 27498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.456837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:41821 (local address 127.0.0.1:41602)
0811 10:13:05.457167 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.457195 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.457254 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.457578 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.457581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.457583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:05.457648 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.457650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.458927 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.458928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.460081 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.460084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.460978 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.460979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.461064 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461077 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.461177 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.461233 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":98,"threads_started":1}
I20260811 10:13:05.461313 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.456939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.8.238.3:41821)
0811 10:13:05.457430 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.457434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.457442 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:13:05.457470 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.457471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.457483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.457547 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:05.457817 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.458795 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.460174 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.460889 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.460971 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461029 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.461196 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.461237 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":99,"threads_started":1}
I20260811 10:13:05.461477 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.456818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:42021 (local address 127.0.0.1:47806)
0811 10:13:05.457144 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.457186 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.457256 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.457983 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.457984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.457986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:05.458030 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.458031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.459389 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.459390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.460366 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.460368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.461246 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.461247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.461309 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461313 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.461364 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.461404 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":60,"threads_started":1}
I20260811 10:13:05.461824 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.456941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.8.238.1:42021)
0811 10:13:05.457724 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.457729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.457739 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:13:05.457783 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.457786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.457802 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.457891 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:05.458113 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.459303 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.460455 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.461093 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.461163 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461210 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.461666 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.461717 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":105,"threads_started":1}
I20260811 10:13:05.462407 27387 tablet_service.cc:1467] Processing CreateTablet for tablet 8690b3193e8b4de59b535d15c7405e54 (DEFAULT_TABLE table=test-table [id=ae6f4f713f2c4c259195e7959a2c992b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:05.462411 26943 tablet_service.cc:1467] Processing CreateTablet for tablet 8690b3193e8b4de59b535d15c7405e54 (DEFAULT_TABLE table=test-table [id=ae6f4f713f2c4c259195e7959a2c992b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:05.462772 26943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8690b3193e8b4de59b535d15c7405e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:05.462772 27387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8690b3193e8b4de59b535d15c7405e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:05.463822 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.457060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:44263 (local address 127.0.0.1:43306)
0811 10:13:05.458131 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.458140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.458186 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.458374 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.458375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.458376 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:05.458412 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.458414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.459646 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.459647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.460687 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.460690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.461674 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.461675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.461730 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461734 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.461783 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.461820 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"thread_start_us":72,"threads_started":1}
I20260811 10:13:05.464157 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.457205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.8.238.2:44263)
0811 10:13:05.457415 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.457419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.458205 (+ 786us) server_negotiation.cc:407] Connection header received
0811 10:13:05.458255 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.458257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.458274 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.458359 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:05.458478 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.459503 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.460780 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.461509 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.461579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.461637 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.464040 (+ 2403us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.464090 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
I20260811 10:13:05.464789 27166 tablet_service.cc:1467] Processing CreateTablet for tablet 8690b3193e8b4de59b535d15c7405e54 (DEFAULT_TABLE table=test-table [id=ae6f4f713f2c4c259195e7959a2c992b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:05.465117 27166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8690b3193e8b4de59b535d15c7405e54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:05.466148 27511 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Bootstrap starting.
I20260811 10:13:05.466382 27512 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: Bootstrap starting.
I20260811 10:13:05.467115 27512 tablet_bootstrap.cc:654] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:05.467768 27512 log.cc:824] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:05.468511 27511 tablet_bootstrap.cc:654] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:05.468616 27512 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: No bootstrap required, opened a new log
I20260811 10:13:05.468690 27512 ts_tablet_manager.cc:1397] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:13:05.469063 27511 log.cc:824] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:05.469720 27511 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: No bootstrap required, opened a new log
I20260811 10:13:05.469774 27511 ts_tablet_manager.cc:1397] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 10:13:05.470335 27512 raft_consensus.cc:348] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.470496 27512 raft_consensus.cc:374] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:05.470546 27512 raft_consensus.cc:729] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec6fb3342c3846a49860af143fddc2f9, State: Initialized, Role: FOLLOWER
I20260811 10:13:05.470669 27512 consensus_queue.cc:260] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.471004 27512 ts_tablet_manager.cc:1428] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:13:05.471004 27050 heartbeater.cc:503] Master 127.8.238.62:35215 was elected leader, sending a full tablet report...
I20260811 10:13:05.471202 27516 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Bootstrap starting.
I20260811 10:13:05.471304 27511 raft_consensus.cc:348] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.471467 27511 raft_consensus.cc:374] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:05.471510 27511 raft_consensus.cc:729] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d419160b6f2744d48f3a6a510754e64a, State: Initialized, Role: FOLLOWER
I20260811 10:13:05.471612 27511 consensus_queue.cc:260] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.471915 27516 tablet_bootstrap.cc:654] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:05.471916 27511 ts_tablet_manager.cc:1428] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:13:05.472074 27486 heartbeater.cc:503] Master 127.8.238.62:35215 was elected leader, sending a full tablet report...
I20260811 10:13:05.472527 27516 log.cc:824] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:05.473285 27516 tablet_bootstrap.cc:492] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: No bootstrap required, opened a new log
I20260811 10:13:05.473369 27516 ts_tablet_manager.cc:1397] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:13:05.474663 27516 raft_consensus.cc:348] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.474803 27516 raft_consensus.cc:374] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:05.474848 27516 raft_consensus.cc:729] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9d65f256d0a4f48988b731cfad28587, State: Initialized, Role: FOLLOWER
I20260811 10:13:05.474953 27516 consensus_queue.cc:260] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.475219 27516 ts_tablet_manager.cc:1428] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:13:05.475250 27278 heartbeater.cc:503] Master 127.8.238.62:35215 was elected leader, sending a full tablet report...
W20260811 10:13:05.502161 27279 tablet.cc:2369] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:13:05.519826 27519 raft_consensus.cc:482] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:13:05.519961 27519 raft_consensus.cc:504] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.520450 27519 leader_election.cc:288] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d419160b6f2744d48f3a6a510754e64a (127.8.238.3:41821), a9d65f256d0a4f48988b731cfad28587 (127.8.238.2:44263)
I20260811 10:13:05.524469 27501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.520757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:55901 (local address 127.8.238.3:41821)
0811 10:13:05.520810 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.520812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.520958 (+ 146us) server_negotiation.cc:407] Connection header received
0811 10:13:05.521010 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.521011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.521021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.521065 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:05.521235 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.521920 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.523437 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.524064 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.524134 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.524146 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.524354 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.524393 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:13:05.524472 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.520718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:52159 (local address 127.8.238.2:44263)
0811 10:13:05.520773 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.520775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.520808 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:13:05.520841 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.520842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.520855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.520906 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:05.521121 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.521914 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.523434 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.524057 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.524135 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.524147 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.524354 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.524393 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:13:05.524531 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.520630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:44263 (local address 127.8.238.1:52159)
0811 10:13:05.520701 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.520716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.520768 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.520936 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.520937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.520939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:05.521026 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.521028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.522002 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.522003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.523341 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.523345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.524147 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.524148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.524238 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.524277 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.524329 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.524372 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":100}
I20260811 10:13:05.524539 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.520637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:41821 (local address 127.8.238.1:55901)
0811 10:13:05.520856 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.520864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:05.520936 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:05.521092 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:05.521093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:05.521095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:05.521160 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.521161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.522002 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.522003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.523348 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:05.523349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.524147 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.524148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:05.524238 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.524245 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:13:05.524340 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 10:13:05.524372 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":13,"thread_start_us":126,"threads_started":1}
I20260811 10:13:05.524866 27200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8690b3193e8b4de59b535d15c7405e54" candidate_uuid: "ec6fb3342c3846a49860af143fddc2f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9d65f256d0a4f48988b731cfad28587" is_pre_election: true
I20260811 10:13:05.525054 27200 raft_consensus.cc:2454] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6fb3342c3846a49860af143fddc2f9 in term 0.
I20260811 10:13:05.525053 27416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8690b3193e8b4de59b535d15c7405e54" candidate_uuid: "ec6fb3342c3846a49860af143fddc2f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d419160b6f2744d48f3a6a510754e64a" is_pre_election: true
I20260811 10:13:05.525220 27416 raft_consensus.cc:2454] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6fb3342c3846a49860af143fddc2f9 in term 0.
I20260811 10:13:05.525302 26869 leader_election.cc:302] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9d65f256d0a4f48988b731cfad28587, ec6fb3342c3846a49860af143fddc2f9; no voters:
I20260811 10:13:05.525467 27519 raft_consensus.cc:2781] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:13:05.525520 27519 raft_consensus.cc:482] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:13:05.525542 27519 raft_consensus.cc:3037] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:05.526301 27519 raft_consensus.cc:504] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.526446 27519 leader_election.cc:288] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [CANDIDATE]: Term 1 election: Requested vote from peers d419160b6f2744d48f3a6a510754e64a (127.8.238.3:41821), a9d65f256d0a4f48988b731cfad28587 (127.8.238.2:44263)
I20260811 10:13:05.526602 27416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8690b3193e8b4de59b535d15c7405e54" candidate_uuid: "ec6fb3342c3846a49860af143fddc2f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d419160b6f2744d48f3a6a510754e64a"
I20260811 10:13:05.526628 27200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8690b3193e8b4de59b535d15c7405e54" candidate_uuid: "ec6fb3342c3846a49860af143fddc2f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9d65f256d0a4f48988b731cfad28587"
I20260811 10:13:05.526697 27200 raft_consensus.cc:3037] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:05.526690 27416 raft_consensus.cc:3037] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:05.527658 27416 raft_consensus.cc:2454] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec6fb3342c3846a49860af143fddc2f9 in term 1.
I20260811 10:13:05.527658 27200 raft_consensus.cc:2454] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec6fb3342c3846a49860af143fddc2f9 in term 1.
I20260811 10:13:05.527833 26869 leader_election.cc:302] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9d65f256d0a4f48988b731cfad28587, ec6fb3342c3846a49860af143fddc2f9; no voters:
I20260811 10:13:05.527948 27519 raft_consensus.cc:2781] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:05.528100 27519 raft_consensus.cc:686] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 1 LEADER]: Becoming Leader. State: Replica: ec6fb3342c3846a49860af143fddc2f9, State: Running, Role: LEADER
I20260811 10:13:05.528179 27519 consensus_queue.cc:237] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } }
I20260811 10:13:05.529093 26749 catalog_manager.cc:5495] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec6fb3342c3846a49860af143fddc2f9 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "ec6fb3342c3846a49860af143fddc2f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec6fb3342c3846a49860af143fddc2f9" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 42021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 } health_report { overall_health: UNKNOWN } } }
I20260811 10:13:05.536903 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.534683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.8.238.1:42021)
0811 10:13:05.534737 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:05.534739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:05.534746 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:13:05.534777 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:05.534778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:05.534801 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:05.534875 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:13:05.535007 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.535438 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.536183 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:05.536374 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:05.536428 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:05.536559 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:13:05.536647 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:13:05.536704 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:05.536741 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 10:13:05.536757 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 10:13:05.538481 27416 raft_consensus.cc:1261] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 1 FOLLOWER]: Refusing update from remote peer ec6fb3342c3846a49860af143fddc2f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:13:05.538483 27200 raft_consensus.cc:1261] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 1 FOLLOWER]: Refusing update from remote peer ec6fb3342c3846a49860af143fddc2f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:13:05.538843 27519 consensus_queue.cc:1035] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9d65f256d0a4f48988b731cfad28587" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 44263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:13:05.538992 27529 consensus_queue.cc:1035] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d419160b6f2744d48f3a6a510754e64a" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 41821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:13:05.541406 27533 mvcc.cc:205] Tried to move back new op lower bound from 7317271287963099136 to 7317271287923998720. Current Snapshot: MvccSnapshot[applied={T|T < 7317271287963099136}]
I20260811 10:13:05.541939 27535 mvcc.cc:205] Tried to move back new op lower bound from 7317271287963099136 to 7317271287923998720. Current Snapshot: MvccSnapshot[applied={T|T < 7317271287963099136}]
I20260811 10:13:05.543967 26748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:13:05.544060 26748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:13:05.544848 26748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9a61816fdd14b44995dbf3e47e25d3e: Sending DeleteTablet for 3 replicas of tablet 8690b3193e8b4de59b535d15c7405e54
I20260811 10:13:05.545224 27387 tablet_service.cc:1514] Processing DeleteTablet for tablet 8690b3193e8b4de59b535d15c7405e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:05 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:41602
I20260811 10:13:05.545224 27166 tablet_service.cc:1514] Processing DeleteTablet for tablet 8690b3193e8b4de59b535d15c7405e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:05 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:43306
I20260811 10:13:05.545265 26943 tablet_service.cc:1514] Processing DeleteTablet for tablet 8690b3193e8b4de59b535d15c7405e54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:05 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47806
I20260811 10:13:05.545488 27550 tablet_replica.cc:330] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: stopping tablet replica
I20260811 10:13:05.545590 27551 tablet_replica.cc:330] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9: stopping tablet replica
I20260811 10:13:05.545686 27551 raft_consensus.cc:2229] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:13:05.545692 27550 raft_consensus.cc:2229] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:13:05.545753 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 26846
I20260811 10:13:05.545812 27551 raft_consensus.cc:2258] T 8690b3193e8b4de59b535d15c7405e54 P ec6fb3342c3846a49860af143fddc2f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:13:05.550227 27550 raft_consensus.cc:2258] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:13:05.551527 27549 tablet_replica.cc:330] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: stopping tablet replica
I20260811 10:13:05.551625 27549 raft_consensus.cc:2229] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:13:05.551707 27549 raft_consensus.cc:2258] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:13:05.552019 27550 ts_tablet_manager.cc:1905] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:13:05.553743 27550 ts_tablet_manager.cc:1918] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:13:05.553819 27550 log.cc:1196] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/8690b3193e8b4de59b535d15c7405e54
I20260811 10:13:05.554064 27550 ts_tablet_manager.cc:1939] T 8690b3193e8b4de59b535d15c7405e54 P a9d65f256d0a4f48988b731cfad28587: Deleting consensus metadata
I20260811 10:13:05.554489 26725 catalog_manager.cc:4841] TS a9d65f256d0a4f48988b731cfad28587 (127.8.238.2:44263): tablet 8690b3193e8b4de59b535d15c7405e54 (table test-table [id=ae6f4f713f2c4c259195e7959a2c992b]) successfully deleted
I20260811 10:13:05.556658 27549 ts_tablet_manager.cc:1905] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:13:05.558939 27549 ts_tablet_manager.cc:1918] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:13:05.559019 27549 log.cc:1196] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/8690b3193e8b4de59b535d15c7405e54
I20260811 10:13:05.559319 27549 ts_tablet_manager.cc:1939] T 8690b3193e8b4de59b535d15c7405e54 P d419160b6f2744d48f3a6a510754e64a: Deleting consensus metadata
I20260811 10:13:05.559809 26731 catalog_manager.cc:4841] TS d419160b6f2744d48f3a6a510754e64a (127.8.238.3:41821): tablet 8690b3193e8b4de59b535d15c7405e54 (table test-table [id=ae6f4f713f2c4c259195e7959a2c992b]) successfully deleted
W20260811 10:13:05.565044 26725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:42021 (error 108)
I20260811 10:13:05.565142 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 27061
I20260811 10:13:05.571671 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:05.571493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:42021 (local address 127.0.0.1:47828)
0811 10:13:05.571572 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:05.571624 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:42021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 10:13:05.571903 26725 catalog_manager.cc:4584] TS ec6fb3342c3846a49860af143fddc2f9 (127.8.238.1:42021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8690b3193e8b4de59b535d15c7405e54: Network error: Client connection negotiation failed: client connection to 127.8.238.1:42021: connect: Connection refused (error 111)
I20260811 10:13:05.576772 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 27284
I20260811 10:13:05.583626 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 26697
2026-08-11T10:13:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2238 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[27559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[27559](info): set up 2 sockets
Aug 11 10:13:05 dist-test-slave-7f01 krb5kdc[27559](info): commencing operation
krb5kdc: starting...
W20260811 10:13:06.873521 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:13:06 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:13:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangeradmin_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangeradmin_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangerlookup_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangerlookup_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
I20260811 10:13:07.045995 9144 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6YJKWE/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/postgres -l logfile start
2026-08-11 10:13:09.591 UTC [27600] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 10:13:09.591 UTC [27600] LOG: listening on IPv4 address "127.8.238.20", port 46143
2026-08-11 10:13:09.594 UTC [27600] LOG: listening on Unix socket "/tmp/.s.PGSQL.46143"
2026-08-11 10:13:09.617 UTC [27604] LOG: database system was shut down at 2026-08-11 10:13:08 UTC
2026-08-11 10:13:09.621 UTC [27600] LOG: database system is ready to accept connections
I20260811 10:13:10.815543 9144 mini_postgres.cc:96] Postgres bound to 46143
2026-08-11 10:13:10.824 UTC [27620] FATAL: database "slave" does not exist
127.8.238.20:46143 - accepting connections
I20260811 10:13:10.824932 9144 mini_ranger.cc:161] Starting Ranger...
I20260811 10:13:10.841325 9144 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 10:13:11.308588 9144 mini_ranger.cc:87] Created ranger Postgres database
I20260811 10:13:11.308681 9144 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin
2026-08-11 10:13:11,613 [I] DB FLAVOR :POSTGRES
2026-08-11 10:13:11,614 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:13:11,614 [I] Checking connection..
2026-08-11 10:13:11,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 10:13:11,883 [I] Checking connection passed.
2026-08-11 10:13:11,883 [I] --------- Verifying version history table ---------
2026-08-11 10:13:11,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:13:12,158 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:13:12,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:13:12,423 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:13:12,423 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 10:13:12,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 10:13:12.685 UTC [27717] WARNING: there is no transaction in progress
2026-08-11 10:13:12,692 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 10:13:12,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:13:12,958 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 10:13:12,959 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:13:12,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 10:13:13,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 10:13:13,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-11 10:13:13,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:13:13,986 [I] Table x_portal_user does not exist in database ranger
2026-08-11 10:13:13,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:13:14,251 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 10:13:14,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:13:14,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:13:14,748 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 10:13:14,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 10:13:15.705 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.720 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.736 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.751 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.767 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.784 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.847 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.854 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.868 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.878 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.891 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.905 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.914 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.920 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.926 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:15.932 UTC [27890] WARNING: there is no transaction in progress
2026-08-11 10:13:16,560 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 10:13:16,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 10:13:16,819 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 10:13:16,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:13:17,084 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 10:13:17,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:13:17,348 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 10:13:17,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:13:17,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:13:17,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-11 10:13:18,121 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 10:13:18,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 10:13:18,374 [I] DB_PATCHES have already been applied
I20260811 10:13:18.380915 9144 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6YJKWE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 10:13:18.380982 9144 mini_ranger.cc:193] Using host: 127.8.238.20
I20260811 10:13:18.385056 9144 mini_ranger.cc:227] Ranger admin URL: http://127.8.238.20:34993
Aug 11, 2026 10:13:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:13:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:13:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:13:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:13:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:13:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:13:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 10:13:19.781416 9144 mini_ranger.cc:160] Time spent starting Ranger: real 8.956s user 0.003s sys 0.007s
Aug 11, 2026 10:13:40 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:13:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:13:40 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:13:40 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:13:41 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 10:13:42.470069 9144 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangerkms_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/rangerkms_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.238.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.238.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 10:13:42 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443222, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 10:13:42.575402 9144 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 10:13:42.587021 9144 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 10:13:43.444568 9144 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 10:13:43.444665 9144 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms
I20260811 10:13:43.444684 9144 mini_ranger_kms.cc:227] Using postgres at 127.8.238.20:46143
2026-08-11 10:13:43,754 [I] DB FLAVOR :POSTGRES
2026-08-11 10:13:43,754 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:13:43,754 [I] Checking connection
2026-08-11 10:13:43,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 10:13:44,039 [I] connection success
2026-08-11 10:13:44,039 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 10:13:44,039 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 10:13:44,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 10:13:44,299 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 10:13:44,299 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:13:44,299 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 10:13:44,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:46143/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 10:13:44,591 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 10:13:45.085954 9144 mini_ranger_kms.cc:310] Created kms service
I20260811 10:13:45.291716 9144 mini_ranger_kms.cc:326] Created kudu user
I20260811 10:13:45.334841 9144 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 10:13:45 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443222, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:45.767520 9144 mini_ranger_kms.cc:384] Added ranger policy
I20260811 10:13:45.767638 9144 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms:/tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 10:13:45.767661 9144 mini_ranger_kms.cc:240] Using host: 127.8.238.20
I20260811 10:13:45.772722 9144 mini_ranger_kms.cc:276] Ranger KMS PID: 28194
I20260811 10:13:45.772835 9144 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.238.20:34287
Aug 11, 2026 10:13:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:13:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:13:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:13:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:13:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 10:13:48 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786443228, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:48.686205 9144 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.111s user 0.000s sys 0.291s
I20260811 10:13:48.686347 9144 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 10:13:48.686441 9144 mini_ranger_kms.cc:401] 127.8.238.20:34287/kms/v1/keys
Aug 11 10:13:48 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443228, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
Aug 11 10:13:49 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443228, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
Aug 11 10:13:51 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:13:51.311209 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44013
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:44013
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:34287/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:51.398496 28273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:51.398701 28273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:51.398723 28273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:51.399926 28273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:13:51.399967 28273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:51.399981 28273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:51.399994 28273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:13:51.400007 28273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:13:51.401568 28273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.238.20:34287/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:44013
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:44013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:51.401834 28273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:51.402101 28273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:51.408103 28284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:51.408110 28285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:51.408210 28273 server_base.cc:1034] running on GCE node
W20260811 10:13:51.408191 28287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:51.408571 28273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:51.408907 28273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:51.410073 28273 hybrid_clock.cc:648] HybridClock initialized: now 1786443231410062 us; error 28 us; skew 500 ppm
Aug 11 10:13:51 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443231, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:51.412936 28273 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:13:51.413444 28273 webserver.cc:460] Webserver started at http://127.8.238.62:32965/ using document root <none> and password file <none>
I20260811 10:13:51.413703 28273 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:51.413763 28273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:51.413924 28273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:13:51 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443231, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:51.474788 28273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406"
format_stamp: "Formatted at 2026-08-11 10:13:51 on dist-test-slave-7f01"
server_key: "14845fa7ba30b369b3aca81f8ca1cedf"
server_key_iv: "bf2e03a162c61125d6c22df4216091aa"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:51.475363 28273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406"
format_stamp: "Formatted at 2026-08-11 10:13:51 on dist-test-slave-7f01"
server_key: "14845fa7ba30b369b3aca81f8ca1cedf"
server_key_iv: "bf2e03a162c61125d6c22df4216091aa"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:51.477576 28273 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 10:13:51.490207 28294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:51.490425 28273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:13:51.490545 28273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406"
format_stamp: "Formatted at 2026-08-11 10:13:51 on dist-test-slave-7f01"
server_key: "14845fa7ba30b369b3aca81f8ca1cedf"
server_key_iv: "bf2e03a162c61125d6c22df4216091aa"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:51.490654 28273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:51.503441 28273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:51.505371 28273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:51.505544 28273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:51.511937 28273 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:44013
I20260811 10:13:51.512018 28382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:44013 every 8 connection(s)
I20260811 10:13:51.512382 28273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:13:51.513253 28387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:51.516317 28387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: Bootstrap starting.
I20260811 10:13:51.517014 28387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:51.517021 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 28273
I20260811 10:13:51.517125 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:13:51.517313 9144 external_mini_cluster.cc:1428] Setting key 3eae758d901a994399868235a68be4f5
I20260811 10:13:51.517398 28387 log.cc:824] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:51.518260 28387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: No bootstrap required, opened a new log
I20260811 10:13:51.520378 28387 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } }
I20260811 10:13:51.520551 28387 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:51.520620 28387 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2cfdeeec3e4d9ea85e42382e4f3406, State: Initialized, Role: FOLLOWER
I20260811 10:13:51.520835 28387 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } }
I20260811 10:13:51.520941 28387 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:13:51.520999 28387 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:13:51.521059 28387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:51.521607 28387 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } }
I20260811 10:13:51.521732 28387 leader_election.cc:302] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec2cfdeeec3e4d9ea85e42382e4f3406; no voters:
I20260811 10:13:51.521953 28387 leader_election.cc:288] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:13:51.522008 28394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:51.522224 28394 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [term 1 LEADER]: Becoming Leader. State: Replica: ec2cfdeeec3e4d9ea85e42382e4f3406, State: Running, Role: LEADER
I20260811 10:13:51.522362 28387 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:13:51.522336 28394 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } }
I20260811 10:13:51.524680 28395 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } } }
I20260811 10:13:51.524765 28395 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:51.524989 28396 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec2cfdeeec3e4d9ea85e42382e4f3406. Latest consensus state: current_term: 1 leader_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec2cfdeeec3e4d9ea85e42382e4f3406" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 44013 } } }
I20260811 10:13:51.525116 28396 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406 [sys.catalog]: This master's current role is: LEADER
I20260811 10:13:51.525127 28411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:13:51.525755 28411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:13:51 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:51.526274 28387 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:13:51.532541 28411 catalog_manager.cc:1347] Generated new cluster ID: ed7c1ac2edae4d0ab3f56bf43e378872
I20260811 10:13:51.532636 28411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:13:51.533852 28393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:51.517890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.8.238.62:44013)
0811 10:13:51.518056 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:51.518065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:51.518125 (+ 60us) server_negotiation.cc:407] Connection header received
0811 10:13:51.518669 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:51.518679 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:51.518743 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:51.518890 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:51.519821 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:51.521676 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:51.522813 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:51.523498 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:51.523614 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:51.526504 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:51.526509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:51.526509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:51.526522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:51.531851 (+ 5329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:51.532334 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:51.532337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:51.532338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:51.532504 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:51.532719 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:51.532721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:51.532721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:51.533165 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:51.533326 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:51.533427 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 10:13:51.533615 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260811 10:13:51.565536 28411 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:13:51.566429 28411 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:13:51.574728 28411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: Generated new TSK 0
I20260811 10:13:51.575006 28411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:13:52.269042 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:34287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:52.386255 28457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:52.386533 28457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:52.386579 28457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:52.387842 28457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:52.387918 28457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:52.387962 28457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:13:52.389485 28457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.238.20:34287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:52.389744 28457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:52.389976 28457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:52.395023 28467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:52.394994 28468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:52.395254 28470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:52.395539 28457 server_base.cc:1034] running on GCE node
I20260811 10:13:52.395723 28457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:52.396008 28457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:52.397153 28457 hybrid_clock.cc:648] HybridClock initialized: now 1786443232397141 us; error 28 us; skew 500 ppm
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:52.400046 28457 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:13:52.400545 28457 webserver.cc:460] Webserver started at http://127.8.238.1:39431/ using document root <none> and password file <none>
I20260811 10:13:52.400791 28457 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:52.400856 28457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:52.400995 28457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:52.421931 28457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "77960550b7ca42efb8f39624f838c766"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "79eba35637999df0bba82b894cfc3076"
server_key_iv: "7b5820fa2ce4b58f4012701edf77494f"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.422603 28457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "77960550b7ca42efb8f39624f838c766"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "79eba35637999df0bba82b894cfc3076"
server_key_iv: "7b5820fa2ce4b58f4012701edf77494f"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.424804 28457 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:13:52.440186 28490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.444345 28457 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260811 10:13:52.444494 28457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "77960550b7ca42efb8f39624f838c766"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "79eba35637999df0bba82b894cfc3076"
server_key_iv: "7b5820fa2ce4b58f4012701edf77494f"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.444658 28457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:52.473924 28457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:52.476202 28457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:52.476389 28457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:52.476672 28457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:52.477074 28457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:52.477130 28457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.477164 28457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:52.477223 28457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.485468 28457 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:32849
I20260811 10:13:52.485538 28675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:32849 every 8 connection(s)
I20260811 10:13:52.485869 28457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:13:52.486099 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 28457
I20260811 10:13:52.486213 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:13:52.486430 9144 external_mini_cluster.cc:1428] Setting key 53c1897c1db3b7da918201a366d61a5c
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:52.498871 28679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.486788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44013 (local address 127.8.238.1:60465)
0811 10:13:52.487020 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:52.487066 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:52.488231 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:52.488643 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:52.488649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:52.488945 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:52.489259 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.489265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.489971 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.489973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.491215 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.491219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.491812 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.491814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.491957 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:52.492581 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:52.492591 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:52.494324 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:52.496559 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:52.496566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:52.496569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:52.496861 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:52.497086 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:52.497087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:52.497087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:52.497233 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:52.498080 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:52.498088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:52.498271 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 10:13:52.498550 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 10:13:52.498700 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":153,"threads_started":1}
I20260811 10:13:52.499346 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.487175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:60465 (local address 127.8.238.62:44013)
0811 10:13:52.487853 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:52.487856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:52.487867 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:13:52.488414 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:52.488415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:52.488430 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:52.488501 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:52.489356 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.489871 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.491321 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.491651 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.491718 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:52.494437 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:52.494439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:52.494439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:52.494447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:52.496462 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:52.496952 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:52.496953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:52.496953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:52.497020 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:52.497321 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:52.497322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:52.497322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:52.497489 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:52.497555 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:52.499168 (+ 1613us) server_negotiation.cc:299] Negotiation successful
0811 10:13:52.499258 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":151,"threads_started":1}
I20260811 10:13:52.499760 28677 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44013
I20260811 10:13:52.499864 28677 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:52.500056 28677 heartbeater.cc:511] Master 127.8.238.62:44013 requested a full tablet report, sending...
I20260811 10:13:52.500617 28330 ts_manager.cc:194] Registered new tserver with Master: 77960550b7ca42efb8f39624f838c766 (127.8.238.1:32849)
I20260811 10:13:52.501508 28330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:60465
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:13:52.532740 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:34287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 10:13:52.656688 28693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:52.656939 28693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:52.656980 28693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:52.658349 28693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:52.658411 28693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:52.658452 28693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:13:52.660154 28693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.238.20:34287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:52.660388 28693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:52.660651 28693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:52.666495 28702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:52.666507 28704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:52.666467 28701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:52.667026 28693 server_base.cc:1034] running on GCE node
I20260811 10:13:52.667224 28693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:52.667466 28693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:52.668645 28693 hybrid_clock.cc:648] HybridClock initialized: now 1786443232668626 us; error 31 us; skew 500 ppm
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:52.671370 28693 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:13:52.671921 28693 webserver.cc:460] Webserver started at http://127.8.238.2:44013/ using document root <none> and password file <none>
I20260811 10:13:52.672144 28693 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:52.672212 28693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:52.672349 28693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:52.691485 28693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "ef02594bfe9d4a0cba659527e7e1d69e"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "c04e155c3487dcd90a5d1bb1ec8e2214"
server_key_iv: "0a8f378077aed2244cbb39e116a9d01a"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.692292 28693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "ef02594bfe9d4a0cba659527e7e1d69e"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "c04e155c3487dcd90a5d1bb1ec8e2214"
server_key_iv: "0a8f378077aed2244cbb39e116a9d01a"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.694907 28693 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 10:13:52.721921 28719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.722198 28693 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260811 10:13:52.722368 28693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "ef02594bfe9d4a0cba659527e7e1d69e"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "c04e155c3487dcd90a5d1bb1ec8e2214"
server_key_iv: "0a8f378077aed2244cbb39e116a9d01a"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.722554 28693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:52.735944 28693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:52.738454 28693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:52.738734 28693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:52.739094 28693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:52.739506 28693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:52.739593 28693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.739663 28693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:52.739725 28693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.750106 28693 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:45545
I20260811 10:13:52.750192 28897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:45545 every 8 connection(s)
I20260811 10:13:52.750663 28693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:52.760195 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 28693
I20260811 10:13:52.760331 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:13:52.760561 9144 external_mini_cluster.cc:1428] Setting key ea643f761eadf6f32077319bc6a4083e
I20260811 10:13:52.761968 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.751637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:42113 (local address 127.8.238.62:44013)
0811 10:13:52.751687 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:52.751690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:52.751987 (+ 297us) server_negotiation.cc:407] Connection header received
0811 10:13:52.752675 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:52.752677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:52.752691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:52.752722 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:52.753555 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.754373 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.755870 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.756195 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.756243 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:52.758802 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:52.758804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:52.758805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:52.758812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:52.760314 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:52.760843 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:52.760845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:52.760845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:52.760915 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:52.761221 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:52.761221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:52.761222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:52.761382 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:52.761431 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:52.761786 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 10:13:52.761877 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:13:52.762171 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.751627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44013 (local address 127.8.238.2:42113)
0811 10:13:52.751846 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:52.751893 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:52.752542 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:52.752925 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:52.752929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:52.753172 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:52.753440 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.753446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.754491 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.754492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.755732 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.755756 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.756341 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.756342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.756484 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:52.757125 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:52.757136 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:52.758697 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:52.760422 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:52.760429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:52.760431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:52.760710 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:52.761011 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:52.761012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:52.761012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:52.761144 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:52.761478 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:52.761485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:52.761649 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 10:13:52.761764 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 10:13:52.761918 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":114,"threads_started":1}
I20260811 10:13:52.762656 28898 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44013
I20260811 10:13:52.762775 28898 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:52.762970 28898 heartbeater.cc:511] Master 127.8.238.62:44013 requested a full tablet report, sending...
I20260811 10:13:52.763384 28330 ts_manager.cc:194] Registered new tserver with Master: ef02594bfe9d4a0cba659527e7e1d69e (127.8.238.2:45545)
I20260811 10:13:52.763914 28330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:42113
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:13:52.791293 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:34287/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:13:52.891647 28905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:13:52.891965 28905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:13:52.892009 28905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:13:52.893491 28905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:13:52.893591 28905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:13:52.893654 28905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:13:52.895576 28905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.238.20:34287/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:44013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:13:52.895871 28905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:13:52.896169 28905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:13:52.902153 28912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:13:52.902180 28911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:52.902383 28905 server_base.cc:1034] running on GCE node
W20260811 10:13:52.902139 28915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:13:52.902733 28905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:13:52.903023 28905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:13:52.904188 28905 hybrid_clock.cc:648] HybridClock initialized: now 1786443232904161 us; error 40 us; skew 500 ppm
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:13:52.907347 28905 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:13:52.907841 28905 webserver.cc:460] Webserver started at http://127.8.238.3:41015/ using document root <none> and password file <none>
I20260811 10:13:52.908084 28905 fs_manager.cc:359] Metadata directory not provided
I20260811 10:13:52.908154 28905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:13:52.908336 28905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:52.923144 28905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "faf7d9cc721f4dbeb49228b4a17c7823"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "d5dbd7ded47c6b71836cabc5828f4178"
server_key_iv: "75fd00a06ef93ef6b8f7af372a35b8ed"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.923920 28905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "faf7d9cc721f4dbeb49228b4a17c7823"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "d5dbd7ded47c6b71836cabc5828f4178"
server_key_iv: "75fd00a06ef93ef6b8f7af372a35b8ed"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.926393 28905 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:13:52.951458 28933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.951725 28905 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260811 10:13:52.951877 28905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "faf7d9cc721f4dbeb49228b4a17c7823"
format_stamp: "Formatted at 2026-08-11 10:13:52 on dist-test-slave-7f01"
server_key: "d5dbd7ded47c6b71836cabc5828f4178"
server_key_iv: "75fd00a06ef93ef6b8f7af372a35b8ed"
server_key_version: "kuduclusterkey@0"
I20260811 10:13:52.951987 28905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:13:52.965883 28905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:13:52.968639 28905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:13:52.968825 28905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:13:52.969204 28905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:13:52.969635 28905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:13:52.969686 28905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.969723 28905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:13:52.969740 28905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:13:52.977938 28905 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:34825
I20260811 10:13:52.978621 28905 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:13:52.988382 29123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:34825 every 8 connection(s)
I20260811 10:13:52.988744 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 28905
I20260811 10:13:52.988886 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:13:52.989164 9144 external_mini_cluster.cc:1428] Setting key fff1fdf4fe56415ba94681efa8a56b52
Aug 11 10:13:52 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:13:53.005831 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.989694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:58471 (local address 127.8.238.62:44013)
0811 10:13:52.989823 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:52.989826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:52.993816 (+ 3990us) server_negotiation.cc:407] Connection header received
0811 10:13:52.994517 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:52.994518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:52.994532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:52.994587 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:52.996760 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.997493 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.998987 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.999293 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.999353 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.002356 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:53.002357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:53.002358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:53.002366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:53.004188 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:53.004697 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:53.004698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:53.004698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:53.004756 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:53.005050 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:53.005051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:53.005052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:53.005188 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:53.005247 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.005644 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.005729 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:13:53.006359 29126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:52.989059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:44013 (local address 127.8.238.3:58471)
0811 10:13:52.993493 (+ 4434us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:52.993546 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:52.994388 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:52.995987 (+ 1599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:52.995992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:52.996262 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:13:52.996637 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.996644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.997613 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.997614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.998873 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:52.998876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:52.999454 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:52.999455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:52.999599 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.000264 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:13:53.000274 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:13:53.002227 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:13:53.004284 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:53.004290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:53.004293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:53.004604 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:53.004826 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:13:53.004826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:13:53.004827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:13:53.004966 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:13:53.005283 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:13:53.005290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:13:53.005477 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.005939 (+ 462us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.006109 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4383,"thread_start_us":281,"threads_started":1}
I20260811 10:13:53.006853 29124 heartbeater.cc:347] Connected to a master server at 127.8.238.62:44013
I20260811 10:13:53.006969 29124 heartbeater.cc:464] Registering TS with master...
I20260811 10:13:53.007164 29124 heartbeater.cc:511] Master 127.8.238.62:44013 requested a full tablet report, sending...
I20260811 10:13:53.007691 28330 ts_manager.cc:194] Registered new tserver with Master: faf7d9cc721f4dbeb49228b4a17c7823 (127.8.238.3:34825)
I20260811 10:13:53.008219 28330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:58471
I20260811 10:13:53.012655 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:13:53.024426 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.014950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.8.238.62:44013)
0811 10:13:53.014989 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.014991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.015263 (+ 272us) server_negotiation.cc:407] Connection header received
0811 10:13:53.015365 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.015366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.015376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.015407 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:13:53.016316 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.017052 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.019307 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.019597 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.019645 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.021294 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:13:53.021296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:13:53.021296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:13:53.021303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:13:53.023073 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:53.023508 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:53.023510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:53.023510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:53.023572 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:13:53.023819 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:13:53.023821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:13:53.023821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:13:53.023954 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:13:53.024031 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.024252 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.024335 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:13:53.026669 28330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:13:53.029119 28330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:13:53.047726 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.8.238.2:45545)
0811 10:13:53.037069 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.037074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.037977 (+ 903us) server_negotiation.cc:407] Connection header received
0811 10:13:53.038022 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.038024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.038040 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.038108 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:53.038290 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.039800 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.045978 (+ 6178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.046816 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.046880 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.046978 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.047584 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.047631 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":98,"threads_started":1}
I20260811 10:13:53.050392 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:45545 (local address 127.0.0.1:40184)
0811 10:13:53.037867 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.037877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:53.037921 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:53.038180 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:53.038181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:53.038182 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:53.038220 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.038222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.041219 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.041219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.042523 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.042526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.047048 (+ 4522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.047049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.047139 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.047154 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.047225 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.050311 (+ 3086us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598,"thread_start_us":80,"threads_started":1}
I20260811 10:13:53.051398 28795 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 (DEFAULT_TABLE table=test-table [id=0a5e89db36324238af69b61774b83c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:53.052245 28795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4c9b95884f4b81854bc0416316d1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:53.057536 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32849 (local address 127.0.0.1:35768)
0811 10:13:53.036688 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.037135 (+ 447us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:53.037202 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:53.037679 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:53.037682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:53.037685 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:53.037752 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.037754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.042825 (+ 5071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.042834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.044396 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.044398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.057248 (+ 12850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.057249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.057338 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.057344 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.057395 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.057443 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":78,"threads_started":1}
I20260811 10:13:53.058023 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.8.238.1:32849)
0811 10:13:53.037448 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.037452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.037462 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:13:53.037498 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.037501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.037517 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.037595 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:53.039918 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.042726 (+ 2808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.044494 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.045272 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.057741 (+ 12469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.057854 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.057890 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.057949 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":65,"threads_started":1}
I20260811 10:13:53.058741 28568 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 (DEFAULT_TABLE table=test-table [id=0a5e89db36324238af69b61774b83c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:53.059129 28568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4c9b95884f4b81854bc0416316d1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:53.064957 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34825 (local address 127.0.0.1:49574)
0811 10:13:53.045391 (+ 8963us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.045401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:53.045454 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:53.048148 (+ 2694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:53.048150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:53.048152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:53.048194 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.048195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.049553 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.049554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.063479 (+ 13925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.063484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.064683 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.064684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.064758 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.064764 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.064818 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.064875 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8944,"thread_start_us":74,"threads_started":1}
I20260811 10:13:53.064962 29149 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Bootstrap starting.
I20260811 10:13:53.065371 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.036894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.8.238.3:34825)
0811 10:13:53.037376 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.037380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.045610 (+ 8230us) server_negotiation.cc:407] Connection header received
0811 10:13:53.045746 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.045749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.045768 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.045850 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:53.048280 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.049453 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.063669 (+ 14216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.064592 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.064679 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.064762 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.065236 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.065282 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":88,"threads_started":1}
I20260811 10:13:53.066161 29149 tablet_bootstrap.cc:654] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:53.066318 29006 tablet_service.cc:1467] Processing CreateTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 (DEFAULT_TABLE table=test-table [id=0a5e89db36324238af69b61774b83c91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:13:53.066596 29149 log.cc:824] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:53.066659 29006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b4c9b95884f4b81854bc0416316d1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:13:53.067018 29150 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: Bootstrap starting.
I20260811 10:13:53.067927 29150 tablet_bootstrap.cc:654] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:53.068351 29150 log.cc:824] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:53.069152 29149 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: No bootstrap required, opened a new log
I20260811 10:13:53.069216 29149 ts_tablet_manager.cc:1397] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 10:13:53.071429 29149 raft_consensus.cc:348] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.071610 29149 raft_consensus.cc:374] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:53.071658 29149 raft_consensus.cc:729] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef02594bfe9d4a0cba659527e7e1d69e, State: Initialized, Role: FOLLOWER
I20260811 10:13:53.071805 29149 consensus_queue.cc:260] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.072109 29149 ts_tablet_manager.cc:1428] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:13:53.072237 28898 heartbeater.cc:503] Master 127.8.238.62:44013 was elected leader, sending a full tablet report...
I20260811 10:13:53.073235 29150 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: No bootstrap required, opened a new log
I20260811 10:13:53.073318 29150 ts_tablet_manager.cc:1397] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260811 10:13:53.076172 29150 raft_consensus.cc:348] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.076344 29150 raft_consensus.cc:374] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:53.076396 29150 raft_consensus.cc:729] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77960550b7ca42efb8f39624f838c766, State: Initialized, Role: FOLLOWER
I20260811 10:13:53.076540 29150 consensus_queue.cc:260] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.076802 29150 ts_tablet_manager.cc:1428] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 10:13:53.076944 28677 heartbeater.cc:503] Master 127.8.238.62:44013 was elected leader, sending a full tablet report...
I20260811 10:13:53.077669 29155 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Bootstrap starting.
I20260811 10:13:53.078703 29155 tablet_bootstrap.cc:654] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Neither blocks nor log segments found. Creating new log.
I20260811 10:13:53.079036 29155 log.cc:824] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Log is configured to *not* fsync() on all Append() calls
I20260811 10:13:53.080909 29155 tablet_bootstrap.cc:492] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: No bootstrap required, opened a new log
I20260811 10:13:53.080987 29155 ts_tablet_manager.cc:1397] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:13:53.082980 29155 raft_consensus.cc:348] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.083159 29155 raft_consensus.cc:374] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:13:53.083204 29155 raft_consensus.cc:729] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf7d9cc721f4dbeb49228b4a17c7823, State: Initialized, Role: FOLLOWER
I20260811 10:13:53.083346 29155 consensus_queue.cc:260] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.083602 29155 ts_tablet_manager.cc:1428] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:13:53.083716 29124 heartbeater.cc:503] Master 127.8.238.62:44013 was elected leader, sending a full tablet report...
I20260811 10:13:53.089919 29156 raft_consensus.cc:482] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:13:53.090040 29156 raft_consensus.cc:504] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.090415 29156 leader_election.cc:288] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77960550b7ca42efb8f39624f838c766 (127.8.238.1:32849), faf7d9cc721f4dbeb49228b4a17c7823 (127.8.238.3:34825)
I20260811 10:13:53.094532 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.090587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32849 (local address 127.8.238.2:44251)
0811 10:13:53.090812 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.090825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:53.090887 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:53.091179 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:53.091180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:53.091182 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:53.091251 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.091253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.092203 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.092205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.093397 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.093401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.094276 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.094277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.094355 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.094359 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.094417 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.094459 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260811 10:13:53.094936 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.090677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:44251 (local address 127.8.238.1:32849)
0811 10:13:53.090720 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.090723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.090976 (+ 253us) server_negotiation.cc:407] Connection header received
0811 10:13:53.091005 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.091007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.091017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.091060 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:53.091326 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.092102 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.093498 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.094129 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.094200 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.094213 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.094811 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.094868 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:13:53.095193 28603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4c9b95884f4b81854bc0416316d1d9" candidate_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77960550b7ca42efb8f39624f838c766" is_pre_election: true
I20260811 10:13:53.095328 28603 raft_consensus.cc:2454] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef02594bfe9d4a0cba659527e7e1d69e in term 0.
I20260811 10:13:53.095626 28723 leader_election.cc:302] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77960550b7ca42efb8f39624f838c766, ef02594bfe9d4a0cba659527e7e1d69e; no voters:
I20260811 10:13:53.095795 29156 raft_consensus.cc:2781] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:13:53.095865 29156 raft_consensus.cc:482] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:13:53.095899 29156 raft_consensus.cc:3037] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:53.096851 29156 raft_consensus.cc:504] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.096974 29156 leader_election.cc:288] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [CANDIDATE]: Term 1 election: Requested vote from peers 77960550b7ca42efb8f39624f838c766 (127.8.238.1:32849), faf7d9cc721f4dbeb49228b4a17c7823 (127.8.238.3:34825)
I20260811 10:13:53.097251 28603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4c9b95884f4b81854bc0416316d1d9" candidate_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77960550b7ca42efb8f39624f838c766"
I20260811 10:13:53.097328 28603 raft_consensus.cc:3037] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:53.098191 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.090690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34825 (local address 127.8.238.2:43853)
0811 10:13:53.094612 (+ 3922us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.094623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:13:53.094666 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:13:53.094982 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:13:53.094984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:13:53.094985 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:13:53.095018 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.095020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.096069 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.096070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.097136 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:13:53.097140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.097982 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.097983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:13:53.098042 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.098045 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:13:53.098090 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 10:13:53.098122 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3911,"thread_start_us":96,"threads_started":1}
I20260811 10:13:53.098377 28603 raft_consensus.cc:2454] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef02594bfe9d4a0cba659527e7e1d69e in term 1.
I20260811 10:13:53.098428 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.091131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43853 (local address 127.8.238.3:34825)
0811 10:13:53.091181 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.091183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.094710 (+ 3527us) server_negotiation.cc:407] Connection header received
0811 10:13:53.094740 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.094741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.094755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.094804 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:13:53.095097 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.095807 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.097315 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.097833 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.097937 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.097949 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.098331 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.098368 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:13:53.098692 29043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4c9b95884f4b81854bc0416316d1d9" candidate_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" is_pre_election: true
I20260811 10:13:53.098837 29043 raft_consensus.cc:2454] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef02594bfe9d4a0cba659527e7e1d69e in term 0.
I20260811 10:13:53.098906 29042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b4c9b95884f4b81854bc0416316d1d9" candidate_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faf7d9cc721f4dbeb49228b4a17c7823"
I20260811 10:13:53.098987 29042 raft_consensus.cc:3037] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:13:53.099858 29042 raft_consensus.cc:2454] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef02594bfe9d4a0cba659527e7e1d69e in term 1.
I20260811 10:13:53.100195 28723 leader_election.cc:302] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef02594bfe9d4a0cba659527e7e1d69e, faf7d9cc721f4dbeb49228b4a17c7823; no voters:
I20260811 10:13:53.100328 29156 raft_consensus.cc:2781] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:13:53.100484 29156 raft_consensus.cc:686] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 1 LEADER]: Becoming Leader. State: Replica: ef02594bfe9d4a0cba659527e7e1d69e, State: Running, Role: LEADER
I20260811 10:13:53.100574 29156 consensus_queue.cc:237] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } }
I20260811 10:13:53.105005 28330 catalog_manager.cc:5495] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e reported cstate change: term changed from 0 to 1, leader changed from <none> to ef02594bfe9d4a0cba659527e7e1d69e (127.8.238.2). New cstate: current_term: 1 leader_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef02594bfe9d4a0cba659527e7e1d69e" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45545 } health_report { overall_health: HEALTHY } } }
I20260811 10:13:53.115082 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.110617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40186 (local address 127.8.238.2:45545)
0811 10:13:53.110657 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:13:53.110659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:13:53.110668 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:13:53.110755 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:13:53.110756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:13:53.110776 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:13:53.110834 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:13:53.112099 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.112809 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.113913 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:13:53.114214 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:13:53.114279 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:13:53.114511 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:13:53.114632 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:13:53.114712 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:13:53.114975 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 10:13:53.114996 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:13:53.117197 29042 raft_consensus.cc:1261] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 1 FOLLOWER]: Refusing update from remote peer ef02594bfe9d4a0cba659527e7e1d69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:13:53.117579 29164 consensus_queue.cc:1035] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "faf7d9cc721f4dbeb49228b4a17c7823" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:13:53.119678 28603 raft_consensus.cc:1261] T 6b4c9b95884f4b81854bc0416316d1d9 P 77960550b7ca42efb8f39624f838c766 [term 1 FOLLOWER]: Refusing update from remote peer ef02594bfe9d4a0cba659527e7e1d69e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:13:53.120018 29164 consensus_queue.cc:1035] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [LEADER]: Connected to new peer: Peer: permanent_uuid: "77960550b7ca42efb8f39624f838c766" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:13:53.124229 29169 mvcc.cc:205] Tried to move back new op lower bound from 7317271482845130752 to 7317271482780504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317271482845130752}]
I20260811 10:13:53.124788 29174 mvcc.cc:205] Tried to move back new op lower bound from 7317271482845130752 to 7317271482780504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317271482845130752}]
I20260811 10:13:53.127300 29170 mvcc.cc:205] Tried to move back new op lower bound from 7317271482845130752 to 7317271482780504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317271482845130752}]
I20260811 10:13:53.129776 28330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:13:53.129889 28330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:13:53.130621 28330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec2cfdeeec3e4d9ea85e42382e4f3406: Sending DeleteTablet for 3 replicas of tablet 6b4c9b95884f4b81854bc0416316d1d9
I20260811 10:13:53.131131 28568 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:53 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:35768
I20260811 10:13:53.131366 29006 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:53 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:49574
I20260811 10:13:53.131593 28795 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b4c9b95884f4b81854bc0416316d1d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:13:53 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:40184
I20260811 10:13:53.132314 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 28457
I20260811 10:13:53.150509 29181 tablet_replica.cc:330] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: stopping tablet replica
I20260811 10:13:53.150727 29182 tablet_replica.cc:330] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: stopping tablet replica
I20260811 10:13:53.150924 29182 raft_consensus.cc:2229] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:13:53.151130 29181 raft_consensus.cc:2229] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:13:53.151335 29182 raft_consensus.cc:2258] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:13:53.151461 29181 raft_consensus.cc:2258] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:13:53.152130 29182 ts_tablet_manager.cc:1905] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:13:53.153905 29182 ts_tablet_manager.cc:1918] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:13:53.153980 29182 log.cc:1196] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/6b4c9b95884f4b81854bc0416316d1d9
I20260811 10:13:53.156493 29182 ts_tablet_manager.cc:1939] T 6b4c9b95884f4b81854bc0416316d1d9 P ef02594bfe9d4a0cba659527e7e1d69e: Deleting consensus metadata
I20260811 10:13:53.157146 29181 ts_tablet_manager.cc:1905] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:13:53.158819 29181 ts_tablet_manager.cc:1918] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:13:53.158983 29181 log.cc:1196] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/6b4c9b95884f4b81854bc0416316d1d9
I20260811 10:13:53.159655 29181 ts_tablet_manager.cc:1939] T 6b4c9b95884f4b81854bc0416316d1d9 P faf7d9cc721f4dbeb49228b4a17c7823: Deleting consensus metadata
I20260811 10:13:53.168246 28295 catalog_manager.cc:4841] TS ef02594bfe9d4a0cba659527e7e1d69e (127.8.238.2:45545): tablet 6b4c9b95884f4b81854bc0416316d1d9 (table test-table [id=0a5e89db36324238af69b61774b83c91]) successfully deleted
I20260811 10:13:53.168785 28298 catalog_manager.cc:4841] TS faf7d9cc721f4dbeb49228b4a17c7823 (127.8.238.3:34825): tablet 6b4c9b95884f4b81854bc0416316d1d9 (table test-table [id=0a5e89db36324238af69b61774b83c91]) successfully deleted
W20260811 10:13:53.169142 28298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:32849 (error 108)
I20260811 10:13:53.172113 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 28693
Aug 11 10:13:53 dist-test-slave-7f01 krb5kdc[27559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:13:53.204239 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:13:53.203788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32849 (local address 127.0.0.1:35782)
0811 10:13:53.204125 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 10:13:53.204187 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:32849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308}
W20260811 10:13:53.205441 28298 catalog_manager.cc:4584] TS 77960550b7ca42efb8f39624f838c766 (127.8.238.1:32849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b4c9b95884f4b81854bc0416316d1d9: Network error: Client connection negotiation failed: client connection to 127.8.238.1:32849: connect: Connection refused (error 111)
I20260811 10:13:53.212097 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 28905
I20260811 10:13:53.222211 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 28273
I20260811 10:13:53.237236 9144 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 10:13:53.592316 9144 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 10:13:53.592425 9144 mini_ranger.cc:66] Stopping Ranger...
I20260811 10:13:53.946777 9144 mini_ranger.cc:68] Stopped Ranger
2026-08-11 10:13:53.947 UTC [27600] LOG: received smart shutdown request
2026-08-11 10:13:53.950 UTC [27600] LOG: background worker "logical replication launcher" (PID 27614) exited with exit code 1
2026-08-11 10:13:53.951 UTC [27608] LOG: shutting down
2026-08-11 10:13:54.122 UTC [27600] LOG: database system is shut down
2026-08-11T10:13:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48654 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:13:54 dist-test-slave-7f01 krb5kdc[29199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:13:54 dist-test-slave-7f01 krb5kdc[29199](info): set up 2 sockets
Aug 11 10:13:54 dist-test-slave-7f01 krb5kdc[29199](info): commencing operation
krb5kdc: starting...
W20260811 10:13:55.524121 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:13:55 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:13:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangeradmin_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangeradmin_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangerlookup_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangerlookup_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
I20260811 10:13:55.693965 9144 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6YJKWE/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/postgres -l logfile start
2026-08-11 10:13:58.225 UTC [29250] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 10:13:58.225 UTC [29250] LOG: listening on IPv4 address "127.8.238.20", port 50431
2026-08-11 10:13:58.229 UTC [29250] LOG: listening on Unix socket "/tmp/.s.PGSQL.50431"
2026-08-11 10:13:58.251 UTC [29255] LOG: database system was shut down at 2026-08-11 10:13:56 UTC
2026-08-11 10:13:58.258 UTC [29250] LOG: database system is ready to accept connections
I20260811 10:13:59.490294 9144 mini_postgres.cc:96] Postgres bound to 50431
2026-08-11 10:13:59.497 UTC [29272] FATAL: database "slave" does not exist
127.8.238.20:50431 - accepting connections
I20260811 10:13:59.498374 9144 mini_ranger.cc:161] Starting Ranger...
I20260811 10:13:59.513758 9144 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 10:13:59.978526 9144 mini_ranger.cc:87] Created ranger Postgres database
I20260811 10:13:59.978601 9144 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin
2026-08-11 10:14:00,273 [I] DB FLAVOR :POSTGRES
2026-08-11 10:14:00,273 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:14:00,274 [I] Checking connection..
2026-08-11 10:14:00,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 10:14:00,539 [I] Checking connection passed.
2026-08-11 10:14:00,539 [I] --------- Verifying version history table ---------
2026-08-11 10:14:00,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:14:00,805 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:14:00,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:14:01,086 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 10:14:01,086 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 10:14:01,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 10:14:01.332 UTC [29381] WARNING: there is no transaction in progress
2026-08-11 10:14:01,341 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 10:14:01,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 10:14:01,598 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 10:14:01,598 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:14:01,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 10:14:01,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 10:14:02,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-11 10:14:02,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:14:02,601 [I] Table x_portal_user does not exist in database ranger
2026-08-11 10:14:02,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:14:02,870 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 10:14:02,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:14:03,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:14:03,365 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 10:14:03,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 10:14:04.312 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.327 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.342 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.357 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.374 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.393 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.451 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.458 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.471 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.481 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.491 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.503 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.513 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.519 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.525 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:04.532 UTC [29565] WARNING: there is no transaction in progress
2026-08-11 10:14:05,160 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 10:14:05,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 10:14:05,415 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 10:14:05,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 10:14:05,671 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 10:14:05,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 10:14:05,919 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 10:14:05,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 10:14:06,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 10:14:06,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-11 10:14:06,653 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 10:14:06,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 10:14:06,885 [I] DB_PATCHES have already been applied
I20260811 10:14:06.891422 9144 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6YJKWE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 10:14:06.891491 9144 mini_ranger.cc:193] Using host: 127.8.238.20
I20260811 10:14:06.896116 9144 mini_ranger.cc:227] Ranger admin URL: http://127.8.238.20:50833
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:14:07 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 10:14:08.314060 9144 mini_ranger.cc:160] Time spent starting Ranger: real 8.816s user 0.000s sys 0.011s
Aug 11, 2026 10:14:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:14:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 10:14:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:14:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:14:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 10:14:29.242753 9144 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.238.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.238.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangerkms_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.238.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/rangerkms_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.238.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.238.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.238.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/HTTP_127.8.238.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 10:14:29 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 10:14:29.326525 9144 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 10:14:29.336265 9144 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 10:14:30.198791 9144 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 10:14:30.198868 9144 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms
I20260811 10:14:30.198882 9144 mini_ranger_kms.cc:227] Using postgres at 127.8.238.20:50431
2026-08-11 10:14:30,388 [I] DB FLAVOR :POSTGRES
2026-08-11 10:14:30,388 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 10:14:30,388 [I] Checking connection
2026-08-11 10:14:30,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 10:14:30,623 [I] connection success
2026-08-11 10:14:30,623 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 10:14:30,623 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 10:14:30,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 10:14:30,856 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 10:14:30,857 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 10:14:30,857 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 10:14:30,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.238.20:50431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 10:14:31,124 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 10:14:31.599452 9144 mini_ranger_kms.cc:310] Created kms service
I20260811 10:14:31.823365 9144 mini_ranger_kms.cc:326] Created kudu user
I20260811 10:14:31.867585 9144 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 10:14:31 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:32.273099 9144 mini_ranger_kms.cc:384] Added ranger policy
I20260811 10:14:32.273370 9144 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms:/tmp/dist-test-task6YJKWE/build/release/bin/postgresql.jar:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6YJKWE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 10:14:32.273398 9144 mini_ranger_kms.cc:240] Using host: 127.8.238.20
I20260811 10:14:32.278087 9144 mini_ranger_kms.cc:276] Ranger KMS PID: 29859
I20260811 10:14:32.278170 9144 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.238.20:52539
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:14:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 10:14:34 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786443274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 10:14:35 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:35.165562 9144 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.839s user 0.000s sys 0.172s
I20260811 10:14:35.165660 9144 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 10:14:35.165709 9144 mini_ranger_kms.cc:401] 127.8.238.20:52539/kms/v1/keys
Aug 11 10:14:35 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443274, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.238.20@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
Aug 11 10:14:37 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:14:37.676483 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35793
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35793
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:37.761878 29941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:37.762091 29941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:37.762115 29941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:37.763295 29941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:37.763339 29941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:37.763355 29941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:37.763366 29941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:37.763378 29941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:37.765009 29941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.238.20:52539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35793
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:37.765246 29941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:37.765511 29941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:37.770826 29949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:37.770913 29941 server_base.cc:1034] running on GCE node
W20260811 10:14:37.771008 29952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:37.770905 29947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:37.771471 29941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:37.771885 29941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:37.773046 29941 hybrid_clock.cc:648] HybridClock initialized: now 1786443277773035 us; error 37 us; skew 500 ppm
Aug 11 10:14:37 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:37.775830 29941 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:37.776257 29941 webserver.cc:460] Webserver started at http://127.8.238.62:44129/ using document root <none> and password file <none>
I20260811 10:14:37.776499 29941 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:37.776562 29941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:37.776715 29941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:14:37 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:37.839085 29941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "16d2616e33124edf921322180ad3f0bb"
format_stamp: "Formatted at 2026-08-11 10:14:37 on dist-test-slave-7f01"
server_key: "9eab030506c4ce5ff233f7d04ab643ae"
server_key_iv: "19e6191d3c974462e186565fda576d44"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:37.839605 29941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "16d2616e33124edf921322180ad3f0bb"
format_stamp: "Formatted at 2026-08-11 10:14:37 on dist-test-slave-7f01"
server_key: "9eab030506c4ce5ff233f7d04ab643ae"
server_key_iv: "19e6191d3c974462e186565fda576d44"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:37.841256 29941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:37.855541 29963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:37.855846 29941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:37.856009 29941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "16d2616e33124edf921322180ad3f0bb"
format_stamp: "Formatted at 2026-08-11 10:14:37 on dist-test-slave-7f01"
server_key: "9eab030506c4ce5ff233f7d04ab643ae"
server_key_iv: "19e6191d3c974462e186565fda576d44"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:37.856153 29941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:37.863699 29941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:37.865631 29941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:37.865790 29941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:37.870795 29941 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35793
I20260811 10:14:37.870829 30048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35793 every 8 connection(s)
I20260811 10:14:37.871248 29941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:37.872015 30049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:37.873144 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 29941
I20260811 10:14:37.873248 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:14:37.873448 9144 external_mini_cluster.cc:1428] Setting key b481292f2ceee475d819ddfa609c6984
I20260811 10:14:37.874809 30049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Bootstrap starting.
I20260811 10:14:37.875674 30049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:37.876092 30049 log.cc:824] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:37.877261 30049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: No bootstrap required, opened a new log
Aug 11 10:14:37 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:37.881428 30049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:37.881688 30049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:37.881794 30049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d2616e33124edf921322180ad3f0bb, State: Initialized, Role: FOLLOWER
I20260811 10:14:37.882153 30049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:37.882443 30049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:37.882622 30049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:37.882799 30049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:37.883899 30049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:37.884162 30049 leader_election.cc:302] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d2616e33124edf921322180ad3f0bb; no voters:
I20260811 10:14:37.884553 30049 leader_election.cc:288] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:14:37.885071 30049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:37.886459 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:37.873979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.8.238.62:35793)
0811 10:14:37.874243 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:37.874253 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:37.874291 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:14:37.874841 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:37.874850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:37.874923 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:37.875071 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:37.875978 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:37.877481 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:37.878583 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:37.879197 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:37.882343 (+ 3146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:37.882443 (+ 100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:37.882446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:37.882446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:37.882456 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:37.884462 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:37.884871 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:37.884874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:37.884875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:37.884970 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:37.885171 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:37.885172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:37.885172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:37.885457 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:37.886067 (+ 610us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:37.886140 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 10:14:37.886258 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":90,"threads_started":1}
I20260811 10:14:37.886523 30055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:37.886900 30055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 LEADER]: Becoming Leader. State: Replica: 16d2616e33124edf921322180ad3f0bb, State: Running, Role: LEADER
I20260811 10:14:37.887004 30055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:37.887328 30055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d2616e33124edf921322180ad3f0bb. Latest consensus state: current_term: 1 leader_uuid: "16d2616e33124edf921322180ad3f0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } } }
I20260811 10:14:37.887403 30055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:37.887548 30069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16d2616e33124edf921322180ad3f0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } } }
I20260811 10:14:37.887609 30069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:37.887758 30049 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:14:37.887856 30072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:37.890684 30072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:37.895061 30078 mvcc.cc:205] Tried to move back new op lower bound from 7317271666255712256 to 7317271666225569792. Current Snapshot: MvccSnapshot[applied={T|T < 7317271666255712256}]
I20260811 10:14:37.895592 30072 catalog_manager.cc:1347] Generated new cluster ID: 383b59a933e548e38c5c8f8a7f242ffc
I20260811 10:14:37.895646 30072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:37.917613 30072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:14:37.918391 30072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:37.922045 30072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Generated new TSK 0
I20260811 10:14:37.922219 30072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:14:38.643610 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:38.749488 30115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:38.749727 30115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:38.749768 30115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:38.751041 30115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:38.751111 30115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:38.751153 30115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:14:38.752820 30115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:38.753077 30115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:38.753312 30115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:38.759421 30129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:38.759429 30126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:38.759702 30115 server_base.cc:1034] running on GCE node
W20260811 10:14:38.759461 30125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:38.759966 30115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:38.760238 30115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:38.761464 30115 hybrid_clock.cc:648] HybridClock initialized: now 1786443278761442 us; error 36 us; skew 500 ppm
Aug 11 10:14:38 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443278, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:38.764917 30115 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:14:38.765439 30115 webserver.cc:460] Webserver started at http://127.8.238.1:40095/ using document root <none> and password file <none>
I20260811 10:14:38.765697 30115 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:38.765743 30115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:38.765873 30115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:14:38 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443278, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:38.785136 30115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "d6676d98905b4532bceb403ee80e1776"
format_stamp: "Formatted at 2026-08-11 10:14:38 on dist-test-slave-7f01"
server_key: "44fec36f675e264642f1a0a0ec6e741f"
server_key_iv: "b86613c8580ed7ffa06949b9f79918f9"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:38.785751 30115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "d6676d98905b4532bceb403ee80e1776"
format_stamp: "Formatted at 2026-08-11 10:14:38 on dist-test-slave-7f01"
server_key: "44fec36f675e264642f1a0a0ec6e741f"
server_key_iv: "b86613c8580ed7ffa06949b9f79918f9"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:38.787907 30115 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:38.805047 30148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:38.805327 30115 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 10:14:38.805475 30115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "d6676d98905b4532bceb403ee80e1776"
format_stamp: "Formatted at 2026-08-11 10:14:38 on dist-test-slave-7f01"
server_key: "44fec36f675e264642f1a0a0ec6e741f"
server_key_iv: "b86613c8580ed7ffa06949b9f79918f9"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:38.805595 30115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:38.820353 30115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:38.822880 30115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:38.823038 30115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:38.823288 30115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:38.823683 30115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:38.823755 30115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:38.823797 30115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:38.823815 30115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:38.830543 30115 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34765
I20260811 10:14:38.830977 30115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:14:38.833297 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 30115
I20260811 10:14:38.833395 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:14:38.833597 9144 external_mini_cluster.cc:1428] Setting key 6ed4e9454d740c6c68db8a8ac6445e35
I20260811 10:14:38.835506 30348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34765 every 8 connection(s)
Aug 11 10:14:38 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443278, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:38.847925 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:38.836149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.1:45039)
0811 10:14:38.836609 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:38.836655 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:38.837308 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:38.837707 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:38.837712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:38.837976 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:38.838271 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:38.838279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:38.839375 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:38.839377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:38.840745 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:38.840749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:38.841304 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:38.841305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:38.841464 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:38.842075 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:38.842087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:38.843711 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:38.846022 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:38.846029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:38.846032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:38.846352 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:38.846604 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:38.846605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:38.846606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:38.846781 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:38.847133 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:38.847141 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:38.847341 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 10:14:38.847476 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 10:14:38.847656 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":78,"threads_started":1}
I20260811 10:14:38.848258 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:38.836141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:45039 (local address 127.8.238.62:35793)
0811 10:14:38.836435 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:38.836437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:38.836740 (+ 303us) server_negotiation.cc:407] Connection header received
0811 10:14:38.837450 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:38.837451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:38.837466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:38.837559 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:38.838388 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:38.839262 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:38.840848 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:38.841147 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:38.841216 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:38.843841 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:38.843842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:38.843843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:38.843851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:38.845914 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:38.846456 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:38.846457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:38.846458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:38.846530 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:38.846867 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:38.846868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:38.846868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:38.847034 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:38.847101 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:38.848081 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 10:14:38.848164 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":87,"threads_started":1}
I20260811 10:14:38.848721 30350 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:38.848840 30350 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:38.849057 30350 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:38.849658 30001 ts_manager.cc:194] Registered new tserver with Master: d6676d98905b4532bceb403ee80e1776 (127.8.238.1:34765)
I20260811 10:14:38.850524 30001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:45039
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:14:38.880589 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 10:14:39.022375 30361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:39.022643 30361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:39.022676 30361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:39.024366 30361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:39.024422 30361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:39.024459 30361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:14:39.026553 30361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:39.026840 30361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:39.027108 30361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:39.033743 30374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.034021 30376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.034284 30372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:39.035311 30361 server_base.cc:1034] running on GCE node
I20260811 10:14:39.035535 30361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:39.035845 30361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:39.037011 30361 hybrid_clock.cc:648] HybridClock initialized: now 1786443279036989 us; error 34 us; skew 500 ppm
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:39.045487 30361 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:14:39.046000 30361 webserver.cc:460] Webserver started at http://127.8.238.2:39575/ using document root <none> and password file <none>
I20260811 10:14:39.046284 30361 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:39.046345 30361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:39.046484 30361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:39.060518 30361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "aa00712eca12447ba1d68c9aa5178d6b"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "e27db5e915e7845bbdfa2579af824c89"
server_key_iv: "22fff00e29b50ebb8696898912547944"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.060981 30361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "aa00712eca12447ba1d68c9aa5178d6b"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "e27db5e915e7845bbdfa2579af824c89"
server_key_iv: "22fff00e29b50ebb8696898912547944"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.068754 30361 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260811 10:14:39.082180 30396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.082365 30361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:39.082489 30361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "aa00712eca12447ba1d68c9aa5178d6b"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "e27db5e915e7845bbdfa2579af824c89"
server_key_iv: "22fff00e29b50ebb8696898912547944"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.082602 30361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:39.122722 30361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:39.125456 30361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:39.125626 30361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:39.125902 30361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:39.126302 30361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:39.126356 30361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.126415 30361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:39.126446 30361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.136302 30361 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:39549
I20260811 10:14:39.136802 30361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:14:39.137444 30585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:39549 every 8 connection(s)
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:39.145324 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 30361
I20260811 10:14:39.145439 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:14:39.145630 9144 external_mini_cluster.cc:1428] Setting key c8579fc33fcdae7197d00f5385a866a3
I20260811 10:14:39.150803 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.138696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:59043 (local address 127.8.238.62:35793)
0811 10:14:39.138746 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:39.138748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:39.138988 (+ 240us) server_negotiation.cc:407] Connection header received
0811 10:14:39.139670 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:39.139671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:39.139685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:39.139768 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:39.140347 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.141121 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.142259 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.142478 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.142559 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.144670 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:39.144672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:39.144672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:39.144680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:39.149078 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.149608 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.149609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.149610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.149673 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.149933 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.149934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.149934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.150086 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:39.150149 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:39.150542 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 10:14:39.150643 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:14:39.151763 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.138701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.2:59043)
0811 10:14:39.138860 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:39.138905 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:39.139546 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:39.139834 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:39.139845 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:39.140044 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:39.140241 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.140246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.141240 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.141242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.142156 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.142159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.142569 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.142570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.142709 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.143257 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:39.143265 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:39.144563 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:39.149179 (+ 4616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.149185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.149188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.149505 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.149756 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.149756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.149756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.149851 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.150178 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:39.150185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:39.150366 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 10:14:39.151238 (+ 872us) client_negotiation.cc:240] Negotiation successful
0811 10:14:39.151437 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260811 10:14:39.152597 30586 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:39.152693 30586 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:39.152882 30586 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:39.153347 30001 ts_manager.cc:194] Registered new tserver with Master: aa00712eca12447ba1d68c9aa5178d6b (127.8.238.2:39549)
I20260811 10:14:39.153894 30001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:59043
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:14:39.195114 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:39.320108 30602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:39.320353 30602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:39.320389 30602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:39.321625 30602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:39.321697 30602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:39.321743 30602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:14:39.323287 30602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:39.323566 30602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:39.323868 30602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:39.328989 30611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.329172 30615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.329438 30610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:39.329627 30602 server_base.cc:1034] running on GCE node
I20260811 10:14:39.329898 30602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:39.330180 30602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:39.331323 30602 hybrid_clock.cc:648] HybridClock initialized: now 1786443279331316 us; error 30 us; skew 500 ppm
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:39.334030 30602 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:14:39.334539 30602 webserver.cc:460] Webserver started at http://127.8.238.3:39491/ using document root <none> and password file <none>
I20260811 10:14:39.334807 30602 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:39.334887 30602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:39.335040 30602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:39.347831 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "12bc75804d4d402fbd4c52639c53c5bf"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "4d8f718d3f11bf8c78bd4ad6137ad23d"
server_key_iv: "e91967bccc52be6737af043831a63a69"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.348467 30602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "12bc75804d4d402fbd4c52639c53c5bf"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "4d8f718d3f11bf8c78bd4ad6137ad23d"
server_key_iv: "e91967bccc52be6737af043831a63a69"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.358355 30602 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260811 10:14:39.379904 30626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.380131 30602 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260811 10:14:39.380247 30602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "12bc75804d4d402fbd4c52639c53c5bf"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "4d8f718d3f11bf8c78bd4ad6137ad23d"
server_key_iv: "e91967bccc52be6737af043831a63a69"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.380378 30602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:39.395284 30602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:39.397843 30602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:39.397998 30602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:39.398206 30602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:39.398622 30602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:39.398671 30602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.398708 30602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:39.398736 30602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.406939 30602 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:34907
I20260811 10:14:39.407408 30602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:14:39.409051 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 30602
I20260811 10:14:39.409155 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:14:39.409418 9144 external_mini_cluster.cc:1428] Setting key 67a55ba7153b95a6529760fc3950f817
I20260811 10:14:39.410861 30796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:34907 every 8 connection(s)
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:39.423527 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.411513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:55543 (local address 127.8.238.62:35793)
0811 10:14:39.411556 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:39.411559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:39.412053 (+ 494us) server_negotiation.cc:407] Connection header received
0811 10:14:39.412794 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:39.412796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:39.412824 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:39.412858 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:39.413753 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.414538 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.416079 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.416397 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.416445 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.419248 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:39.419250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:39.419250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:39.419258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:39.421183 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.422354 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.422356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.422356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.422422 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.422721 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.422722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.422722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.422871 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:39.422923 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:39.423341 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 10:14:39.423427 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:14:39.424381 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.411530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.3:55543)
0811 10:14:39.411727 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:39.411820 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:39.412658 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:39.413061 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:39.413066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:39.413336 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:39.413636 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.413643 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.414663 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.414664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.415972 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.415975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.416551 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.416552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.416692 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.417323 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:39.417346 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:39.419138 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:39.421902 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.421909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.421912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.422253 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.422499 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.422499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.422500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.422639 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.422978 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:39.422985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:39.423173 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 10:14:39.423951 (+ 778us) client_negotiation.cc:240] Negotiation successful
0811 10:14:39.424138 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260811 10:14:39.425138 30797 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:39.425243 30797 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:39.425446 30797 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:39.425894 30001 ts_manager.cc:194] Registered new tserver with Master: 12bc75804d4d402fbd4c52639c53c5bf (127.8.238.3:34907)
I20260811 10:14:39.426429 30001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:55543
I20260811 10:14:39.432508 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:14:39.433282 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 30115
I20260811 10:14:39.476203 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 30361
I20260811 10:14:39.512446 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 30602
I20260811 10:14:39.525549 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 29941
I20260811 10:14:39.543808 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35793
--webserver_interface=127.8.238.62
--webserver_port=44129
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35793
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:39.629801 30802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:39.630028 30802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:39.630049 30802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:39.631196 30802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:39.631238 30802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:39.631253 30802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:39.631265 30802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:39.631278 30802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:39.632848 30802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35793
--ranger_config_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=44129
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:39.633101 30802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:39.633343 30802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:39.638218 30808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.638218 30810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:39.638463 30802 server_base.cc:1034] running on GCE node
W20260811 10:14:39.638296 30818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:39.638921 30802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:39.639262 30802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:39.640432 30802 hybrid_clock.cc:648] HybridClock initialized: now 1786443279640414 us; error 37 us; skew 500 ppm
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:39.642999 30802 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:39.643505 30802 webserver.cc:460] Webserver started at http://127.8.238.62:44129/ using document root <none> and password file <none>
I20260811 10:14:39.643786 30802 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:39.643853 30802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:39.690769 30802 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:39.691563 30829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.691764 30802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:39.691861 30802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "16d2616e33124edf921322180ad3f0bb"
format_stamp: "Formatted at 2026-08-11 10:14:37 on dist-test-slave-7f01"
server_key: "9eab030506c4ce5ff233f7d04ab643ae"
server_key_iv: "19e6191d3c974462e186565fda576d44"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.692251 30802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:39.710044 30802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:39.711835 30802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:39.711998 30802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:39.716296 30802 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35793
I20260811 10:14:39.716343 30918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35793 every 8 connection(s)
I20260811 10:14:39.716715 30802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:39.717958 30919 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:14:39.718649 30919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Bootstrap starting.
I20260811 10:14:39.720546 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 30802
I20260811 10:14:39.720892 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:34765
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:35793
--webserver_port=40095
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:39.728152 30919 log.cc:824] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:39.730010 30919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:14:39.730325 30919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Bootstrap complete.
I20260811 10:14:39.732818 30919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:39.733058 30919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d2616e33124edf921322180ad3f0bb, State: Initialized, Role: FOLLOWER
I20260811 10:14:39.733235 30919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:39.733310 30919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:39.733361 30919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:39.733410 30919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:14:39.734478 30919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:39.734598 30919 leader_election.cc:302] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d2616e33124edf921322180ad3f0bb; no voters:
I20260811 10:14:39.734812 30919 leader_election.cc:288] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:14:39.734850 30928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:14:39.734977 30928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [term 2 LEADER]: Becoming Leader. State: Replica: 16d2616e33124edf921322180ad3f0bb, State: Running, Role: LEADER
I20260811 10:14:39.735110 30928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } }
I20260811 10:14:39.735118 30919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:39.735407 30932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d2616e33124edf921322180ad3f0bb. Latest consensus state: current_term: 2 leader_uuid: "16d2616e33124edf921322180ad3f0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } } }
I20260811 10:14:39.735517 30932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:39.735855 30938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:39.735407 30933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "16d2616e33124edf921322180ad3f0bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d2616e33124edf921322180ad3f0bb" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35793 } } }
I20260811 10:14:39.735973 30933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:39.736421 30938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:39.736557 30938 catalog_manager.cc:1259] Loaded cluster ID: 383b59a933e548e38c5c8f8a7f242ffc
I20260811 10:14:39.736595 30938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:39.737203 30919 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 10:14:39.737739 30938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:39.740049 30938 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Loaded TSK: 0
I20260811 10:14:39.740890 30938 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:14:39.836928 30925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:39.837220 30925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:39.837258 30925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:39.838905 30925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:39.838976 30925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:39.839017 30925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:14:39.841176 30925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:34765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=40095
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:39.841414 30925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:39.841668 30925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:39.848079 30984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.848307 30980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:39.848150 30982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:39.848789 30925 server_base.cc:1034] running on GCE node
I20260811 10:14:39.849028 30925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:39.849305 30925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:39.850500 30925 hybrid_clock.cc:648] HybridClock initialized: now 1786443279850521 us; error 65 us; skew 500 ppm
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:39.854130 30925 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:14:39.854651 30925 webserver.cc:460] Webserver started at http://127.8.238.1:40095/ using document root <none> and password file <none>
I20260811 10:14:39.854976 30925 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:39.855072 30925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:39.876948 30925 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 10:14:39.878862 30994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.879155 30925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260811 10:14:39.879382 30925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "d6676d98905b4532bceb403ee80e1776"
format_stamp: "Formatted at 2026-08-11 10:14:38 on dist-test-slave-7f01"
server_key: "44fec36f675e264642f1a0a0ec6e741f"
server_key_iv: "b86613c8580ed7ffa06949b9f79918f9"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:39.880239 30925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:39.899578 30925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:39.902119 30925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:39.902294 30925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:39.902576 30925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:39.902997 30925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:39.903051 30925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.903084 30925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:39.903110 30925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:39.910820 30925 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34765
I20260811 10:14:39.911347 30925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:14:39.912034 31189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34765 every 8 connection(s)
I20260811 10:14:39.914772 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 30925
I20260811 10:14:39.915102 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:39549
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:35793
--webserver_port=39575
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:14:39 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:39.927295 31194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.912136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.1:48517)
0811 10:14:39.912288 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:39.912332 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:39.912960 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:39.916050 (+ 3090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:39.916054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:39.916295 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:39.916563 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.916569 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.918204 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.918206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.919562 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:39.919565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.920457 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.920458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:39.920609 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.921229 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:39.921238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:39.922905 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:39.925349 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.925355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.925358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.925655 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.925946 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:39.925947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:39.925947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:39.926082 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:39.926557 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:39.926563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:39.926740 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 10:14:39.926887 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 10:14:39.927049 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260811 10:14:39.928010 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:39.912566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:48517 (local address 127.8.238.62:35793)
0811 10:14:39.915389 (+ 2823us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:39.915399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:39.915442 (+ 43us) server_negotiation.cc:407] Connection header received
0811 10:14:39.915590 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:39.915597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:39.915652 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:39.915897 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:39.916694 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.918087 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.919667 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:39.920328 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:39.920440 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:39.923037 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:39.923041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:39.923041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:39.923054 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:39.925233 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.925753 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.925757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.925757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.925871 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:39.926151 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:39.926152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:39.926152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:39.926444 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:39.926516 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:39.927629 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0811 10:14:39.927807 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2555,"thread_start_us":96,"threads_started":1}
I20260811 10:14:39.928658 31190 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:39.928768 31190 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:39.928973 31190 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:39.929855 31190 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:14:39.929937 31190 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:39.930006 31190 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:39.930342 31190 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:39.930426 31190 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:39.930718 31190 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:14:40.043727 31198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:40.044037 31198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:40.044080 31198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:40.045805 31198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:40.045871 31198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:40.045909 31198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:14:40.048122 31198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:39549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=39575
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:40.048386 31198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:40.048658 31198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:40.054924 31207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:40.054966 31209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:40.054966 31205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:40.055249 31198 server_base.cc:1034] running on GCE node
I20260811 10:14:40.055467 31198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:40.055730 31198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:40.056931 31198 hybrid_clock.cc:648] HybridClock initialized: now 1786443280056902 us; error 41 us; skew 500 ppm
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:40.060029 31198 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:14:40.060529 31198 webserver.cc:460] Webserver started at http://127.8.238.2:39575/ using document root <none> and password file <none>
I20260811 10:14:40.060781 31198 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:40.060875 31198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:40.077412 31198 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:40.078493 31222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.078668 31198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:40.078760 31198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "aa00712eca12447ba1d68c9aa5178d6b"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "e27db5e915e7845bbdfa2579af824c89"
server_key_iv: "22fff00e29b50ebb8696898912547944"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:40.079281 31198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:40.111271 31198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:40.116453 31198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:40.116645 31198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:40.116952 31198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:40.117381 31198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:40.117439 31198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.117482 31198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:40.117507 31198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.125409 31198 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:39549
I20260811 10:14:40.125502 31412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:39549 every 8 connection(s)
I20260811 10:14:40.125854 31198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:14:40.126237 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 31198
I20260811 10:14:40.126565 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.238.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:34907
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:35793
--webserver_port=39491
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:40.140975 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:40.126879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:58629 (local address 127.8.238.62:35793)
0811 10:14:40.126940 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:40.126942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:40.127420 (+ 478us) server_negotiation.cc:407] Connection header received
0811 10:14:40.128286 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:40.128288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:40.128303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:40.128335 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:40.129249 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.130046 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.131583 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.131910 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.131961 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:40.134763 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:40.134765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:40.134766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:40.134774 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:40.136636 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.139729 (+ 3093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.139731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.139731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.139824 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.140145 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.140146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.140147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.140289 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:40.140338 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:40.140768 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 10:14:40.140869 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:14:40.141844 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:40.126879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.2:58629)
0811 10:14:40.127277 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:40.127329 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:40.128151 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:40.128547 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:40.128552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:40.128810 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:40.129125 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:40.129132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.130172 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.130173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:40.131476 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:40.131479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.132082 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.132083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:40.132224 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:40.132882 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:40.132893 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:40.134646 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:40.139253 (+ 4607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:40.139262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:40.139265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:40.139622 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:40.139913 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:40.139914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:40.139914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:40.140063 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:40.140404 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:40.140412 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:40.140589 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 10:14:40.141378 (+ 789us) client_negotiation.cc:240] Negotiation successful
0811 10:14:40.141559 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":99,"threads_started":1}
I20260811 10:14:40.142622 31413 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:40.142728 31413 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.142930 31413 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:40.143800 31413 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:14:40.143877 31413 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.143939 31413 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:40.144351 31413 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.144430 31413 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:40.144829 31413 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:14:40.242154 31418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:40.242435 31418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:40.242472 31418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:40.244256 31418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:40.244306 31418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:40.244343 31418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:14:40.246644 31418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.238.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:34907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=39491
--tserver_master_addrs=127.8.238.62:35793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:40.246945 31418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:40.247212 31418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:40.253665 31428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:40.253685 31432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:40.254122 31429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:40.254405 31418 server_base.cc:1034] running on GCE node
I20260811 10:14:40.254626 31418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:40.254920 31418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:40.256093 31418 hybrid_clock.cc:648] HybridClock initialized: now 1786443280256072 us; error 40 us; skew 500 ppm
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:40.259663 31418 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:14:40.260336 31418 webserver.cc:460] Webserver started at http://127.8.238.3:39491/ using document root <none> and password file <none>
I20260811 10:14:40.260670 31418 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:40.260815 31418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:40.286053 31418 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:14:40.287210 31447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.287487 31418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:40.287583 31418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "12bc75804d4d402fbd4c52639c53c5bf"
format_stamp: "Formatted at 2026-08-11 10:14:39 on dist-test-slave-7f01"
server_key: "4d8f718d3f11bf8c78bd4ad6137ad23d"
server_key_iv: "e91967bccc52be6737af043831a63a69"
server_key_version: "kuduclusterkey@0"
I20260811 10:14:40.288076 31418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:40.302517 31418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:40.305003 31418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:40.305179 31418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:40.305456 31418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:40.305843 31418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:40.305897 31418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.305948 31418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:40.305969 31418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:40.313762 31418 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:34907
I20260811 10:14:40.313833 31625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:34907 every 8 connection(s)
I20260811 10:14:40.314199 31418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:14:40.317914 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 31418
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:40.324273 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:40.315011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:37677 (local address 127.8.238.62:35793)
0811 10:14:40.315068 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:40.315071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:40.315378 (+ 307us) server_negotiation.cc:407] Connection header received
0811 10:14:40.316037 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:40.316039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:40.316053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:40.316115 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:40.316681 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.317262 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.318723 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.319057 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.319134 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:40.321213 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:40.321215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:40.321215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:40.321223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:40.322724 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.323231 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.323232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.323233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.323294 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.323563 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.323564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.323565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.323688 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:40.323727 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:40.324086 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 10:14:40.324178 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:14:40.324880 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:40.315025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:35793 (local address 127.8.238.3:37677)
0811 10:14:40.315251 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:40.315294 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:40.315918 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:40.316186 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:40.316191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:40.316379 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:40.316580 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:40.316584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.317356 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.317357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:40.318627 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:40.318631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.319161 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.319163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:40.319280 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:40.319845 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:40.319854 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:40.321117 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:40.322830 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:40.322837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:40.322839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:40.323138 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:40.323364 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:40.323365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:40.323365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:40.323497 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:40.323789 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:40.323795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:40.323948 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 10:14:40.324510 (+ 562us) client_negotiation.cc:240] Negotiation successful
0811 10:14:40.324661 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":155,"threads_started":1}
I20260811 10:14:40.328382 31629 heartbeater.cc:347] Connected to a master server at 127.8.238.62:35793
I20260811 10:14:40.328492 31629 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.328680 31629 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:40.329322 31629 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:35793 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 10:14:40.329391 31629 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.329453 31629 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:40.329787 31629 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.329888 31629 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
W20260811 10:14:40.330202 31629 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:14:40.343335 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:40.322894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.8.238.62:35793)
0811 10:14:40.324340 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:40.324342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:40.335085 (+ 10743us) server_negotiation.cc:407] Connection header received
0811 10:14:40.335174 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:40.335175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:40.335186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:40.335238 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:40.336130 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.336710 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.337834 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:40.338132 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:40.338212 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:40.340598 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:40.340599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:40.340600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:40.340607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:40.342166 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.342521 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.342522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.342523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.342593 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:40.342786 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:40.342786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:40.342787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:40.342921 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:40.343101 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:40.343150 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 10:14:40.343234 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1431,"thread_start_us":91,"threads_started":1}
Aug 11 10:14:40 dist-test-slave-7f01 krb5kdc[29199](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:40.930925 31190 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:40.931097 31190 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:40.931561 30860 ts_manager.cc:194] Registered new tserver with Master: d6676d98905b4532bceb403ee80e1776 (127.8.238.1:34765)
I20260811 10:14:40.932633 30860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:48517
I20260811 10:14:41.145036 31413 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:41.145202 31413 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:41.145519 30860 ts_manager.cc:194] Registered new tserver with Master: aa00712eca12447ba1d68c9aa5178d6b (127.8.238.2:39549)
I20260811 10:14:41.146078 30860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:58629
Aug 11 10:14:41 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:41.330395 31629 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:41.330574 31629 heartbeater.cc:511] Master 127.8.238.62:35793 requested a full tablet report, sending...
I20260811 10:14:41.330909 30860 ts_manager.cc:194] Registered new tserver with Master: 12bc75804d4d402fbd4c52639c53c5bf (127.8.238.3:34907)
I20260811 10:14:41.331468 30860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:37677
I20260811 10:14:41.337486 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:14:41.346694 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.338522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.8.238.62:35793)
0811 10:14:41.338816 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.338818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.338825 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:14:41.338849 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.338849 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.338862 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.338935 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:41.339650 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.340474 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.341497 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.341803 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.341886 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.343389 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:41.343390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:41.343390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:41.343398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:41.345298 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:41.345693 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:41.345695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:41.345695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:41.345768 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:41.345997 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:41.345998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:41.345999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:41.346148 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:41.346263 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.346386 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.346476 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":61,"threads_started":1}
I20260811 10:14:41.348836 30860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:14:41.351339 30860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:14:41.361887 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.354610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34907 (local address 127.0.0.1:41750)
0811 10:14:41.355486 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.355527 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:41.355595 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:41.355929 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:41.355932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:41.355934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:41.356017 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.356019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.357617 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.357618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.358908 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.358911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.361580 (+ 2669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.361581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.361661 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.361676 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:14:41.361751 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:14:41.361804 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":91,"threads_started":1}
I20260811 10:14:41.362509 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.355119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41750 (local address 127.8.238.3:34907)
0811 10:14:41.355551 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.355556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.355572 (+ 16us) server_negotiation.cc:407] Connection header received
0811 10:14:41.355675 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.355677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.355692 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.355838 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:41.356111 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.357526 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.359021 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.361405 (+ 2384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.361481 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.361540 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.362112 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.362155 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":62,"threads_started":1}
I20260811 10:14:41.363063 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.354656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34765 (local address 127.0.0.1:58250)
0811 10:14:41.355705 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.355713 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:41.357592 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:41.358010 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:41.358011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:41.358013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:41.358054 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.358055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.359500 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.359501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.360558 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.360560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.362846 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.362847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.362915 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.362919 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:14:41.362962 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 10:14:41.362996 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"thread_start_us":92,"threads_started":1}
I20260811 10:14:41.363211 31522 tablet_service.cc:1467] Processing CreateTablet for tablet 8452e310e1ee481aab21cd02c9c4049e (DEFAULT_TABLE table=test-table [id=a11251c15bde4da9b89c1bd18391577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:41.363435 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.354892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.8.238.1:34765)
0811 10:14:41.355315 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.355319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.355960 (+ 641us) server_negotiation.cc:407] Connection header received
0811 10:14:41.357831 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.357833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.357856 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.357906 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:41.358121 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.359376 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.360659 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.361366 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.361420 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.361476 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.363270 (+ 1794us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.363310 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":62,"threads_started":1}
I20260811 10:14:41.363618 31522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8452e310e1ee481aab21cd02c9c4049e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:41.364291 31048 tablet_service.cc:1467] Processing CreateTablet for tablet 8452e310e1ee481aab21cd02c9c4049e (DEFAULT_TABLE table=test-table [id=a11251c15bde4da9b89c1bd18391577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:41.364609 31048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8452e310e1ee481aab21cd02c9c4049e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:41.368124 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.354858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:39549 (local address 127.0.0.1:48004)
0811 10:14:41.356019 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.356028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:41.356069 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:41.356339 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:41.356340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:41.356342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:41.356381 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.356383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.357655 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.357656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.360724 (+ 3068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.360728 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.367865 (+ 7137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.367866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.367932 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.367937 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:14:41.368012 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:14:41.368052 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":1902,"thread_start_us":67,"threads_started":1}
I20260811 10:14:41.368389 31687 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Bootstrap starting.
I20260811 10:14:41.368427 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.355115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.8.238.2:39549)
0811 10:14:41.355416 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.355420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.356128 (+ 708us) server_negotiation.cc:407] Connection header received
0811 10:14:41.356172 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.356175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.356190 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.356267 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:41.356465 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.357503 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.361548 (+ 4045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.362476 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.362545 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.362785 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.368154 (+ 5369us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.368334 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":68,"threads_started":1}
I20260811 10:14:41.369127 31687 tablet_bootstrap.cc:654] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:41.369148 31310 tablet_service.cc:1467] Processing CreateTablet for tablet 8452e310e1ee481aab21cd02c9c4049e (DEFAULT_TABLE table=test-table [id=a11251c15bde4da9b89c1bd18391577a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:41.369457 31310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8452e310e1ee481aab21cd02c9c4049e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:41.369467 31687 log.cc:824] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:41.370249 31687 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: No bootstrap required, opened a new log
I20260811 10:14:41.370312 31687 ts_tablet_manager.cc:1397] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:14:41.372881 31688 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Bootstrap starting.
I20260811 10:14:41.372902 31687 raft_consensus.cc:348] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.373103 31687 raft_consensus.cc:374] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:41.373159 31687 raft_consensus.cc:729] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12bc75804d4d402fbd4c52639c53c5bf, State: Initialized, Role: FOLLOWER
I20260811 10:14:41.373298 31690 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: Bootstrap starting.
I20260811 10:14:41.373294 31687 consensus_queue.cc:260] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.373540 31688 tablet_bootstrap.cc:654] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:41.373574 31687 ts_tablet_manager.cc:1428] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:14:41.373693 31629 heartbeater.cc:503] Master 127.8.238.62:35793 was elected leader, sending a full tablet report...
I20260811 10:14:41.373956 31688 log.cc:824] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:41.374269 31690 tablet_bootstrap.cc:654] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:41.374698 31690 log.cc:824] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:41.374795 31688 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: No bootstrap required, opened a new log
I20260811 10:14:41.374863 31688 ts_tablet_manager.cc:1397] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:14:41.375478 31690 tablet_bootstrap.cc:492] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: No bootstrap required, opened a new log
I20260811 10:14:41.375594 31690 ts_tablet_manager.cc:1397] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:41.377771 31688 raft_consensus.cc:348] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.377990 31688 raft_consensus.cc:374] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:41.378051 31688 raft_consensus.cc:729] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6676d98905b4532bceb403ee80e1776, State: Initialized, Role: FOLLOWER
I20260811 10:14:41.378312 31688 consensus_queue.cc:260] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.378314 31690 raft_consensus.cc:348] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.378582 31690 raft_consensus.cc:374] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:41.378614 31688 ts_tablet_manager.cc:1428] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 10:14:41.378718 31190 heartbeater.cc:503] Master 127.8.238.62:35793 was elected leader, sending a full tablet report...
I20260811 10:14:41.378794 31690 raft_consensus.cc:729] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa00712eca12447ba1d68c9aa5178d6b, State: Initialized, Role: FOLLOWER
I20260811 10:14:41.379029 31690 consensus_queue.cc:260] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.379338 31690 ts_tablet_manager.cc:1428] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 10:14:41.379449 31413 heartbeater.cc:503] Master 127.8.238.62:35793 was elected leader, sending a full tablet report...
W20260811 10:14:41.412875 31193 tablet.cc:2369] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 10:14:41 dist-test-slave-7f01 krb5kdc[29199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for HTTP/127.8.238.20@KRBTEST.COM
I20260811 10:14:41.521759 31697 raft_consensus.cc:482] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:41.521946 31697 raft_consensus.cc:504] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.522383 31697 leader_election.cc:288] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12bc75804d4d402fbd4c52639c53c5bf (127.8.238.3:34907), d6676d98905b4532bceb403ee80e1776 (127.8.238.1:34765)
I20260811 10:14:41.527436 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.523328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:34865 (local address 127.8.238.1:34765)
0811 10:14:41.523378 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.523380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.523706 (+ 326us) server_negotiation.cc:407] Connection header received
0811 10:14:41.523828 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.523829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.523842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.523871 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:41.524107 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.524834 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.526307 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.526940 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.526989 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.527001 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.527313 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.527358 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:14:41.527704 31704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.522982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34765 (local address 127.8.238.2:34865)
0811 10:14:41.523652 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.523660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:41.523728 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:41.523948 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:41.523949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:41.523951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:41.524034 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.524036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.524953 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.524953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.526201 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.526204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.527148 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.527150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.527221 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.527224 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:14:41.527563 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 10:14:41.527604 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":92,"threads_started":1}
I20260811 10:14:41.528064 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8452e310e1ee481aab21cd02c9c4049e" candidate_uuid: "aa00712eca12447ba1d68c9aa5178d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6676d98905b4532bceb403ee80e1776" is_pre_election: true
I20260811 10:14:41.528249 31112 raft_consensus.cc:2454] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa00712eca12447ba1d68c9aa5178d6b in term 0.
I20260811 10:14:41.528530 31225 leader_election.cc:302] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa00712eca12447ba1d68c9aa5178d6b, d6676d98905b4532bceb403ee80e1776; no voters:
I20260811 10:14:41.528683 31697 raft_consensus.cc:2781] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:14:41.528738 31697 raft_consensus.cc:482] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:14:41.528765 31697 raft_consensus.cc:3037] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:41.528805 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.523240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43651 (local address 127.8.238.3:34907)
0811 10:14:41.523305 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.523307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.523604 (+ 297us) server_negotiation.cc:407] Connection header received
0811 10:14:41.523641 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.523642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.523659 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.523709 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:41.524521 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.525286 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.527830 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.528398 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.528447 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.528457 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.528679 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.528723 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:14:41.529035 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.522615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:34907 (local address 127.8.238.2:43651)
0811 10:14:41.523454 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.523465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:41.523546 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:41.524342 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:41.524344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:41.524346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:41.524424 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.524426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.525421 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.525422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.526623 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:41.526626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.528520 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.528521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:41.528596 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.528600 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:14:41.528902 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 10:14:41.528947 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":173,"threads_started":1}
I20260811 10:14:41.529793 31697 raft_consensus.cc:504] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.529969 31697 leader_election.cc:288] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [CANDIDATE]: Term 1 election: Requested vote from peers 12bc75804d4d402fbd4c52639c53c5bf (127.8.238.3:34907), d6676d98905b4532bceb403ee80e1776 (127.8.238.1:34765)
I20260811 10:14:41.530448 31557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8452e310e1ee481aab21cd02c9c4049e" candidate_uuid: "aa00712eca12447ba1d68c9aa5178d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12bc75804d4d402fbd4c52639c53c5bf" is_pre_election: true
I20260811 10:14:41.530625 31557 raft_consensus.cc:2454] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa00712eca12447ba1d68c9aa5178d6b in term 0.
I20260811 10:14:41.530745 31556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8452e310e1ee481aab21cd02c9c4049e" candidate_uuid: "aa00712eca12447ba1d68c9aa5178d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12bc75804d4d402fbd4c52639c53c5bf"
I20260811 10:14:41.530804 31556 raft_consensus.cc:3037] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:41.531651 31556 raft_consensus.cc:2454] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa00712eca12447ba1d68c9aa5178d6b in term 1.
I20260811 10:14:41.532150 31224 leader_election.cc:302] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12bc75804d4d402fbd4c52639c53c5bf, aa00712eca12447ba1d68c9aa5178d6b; no voters:
I20260811 10:14:41.532380 31112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8452e310e1ee481aab21cd02c9c4049e" candidate_uuid: "aa00712eca12447ba1d68c9aa5178d6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6676d98905b4532bceb403ee80e1776"
I20260811 10:14:41.532466 31112 raft_consensus.cc:3037] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:41.533345 31112 raft_consensus.cc:2454] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa00712eca12447ba1d68c9aa5178d6b in term 1.
I20260811 10:14:41.533726 31697 raft_consensus.cc:2781] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:41.533896 31697 raft_consensus.cc:686] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 1 LEADER]: Becoming Leader. State: Replica: aa00712eca12447ba1d68c9aa5178d6b, State: Running, Role: LEADER
I20260811 10:14:41.534003 31697 consensus_queue.cc:237] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } }
I20260811 10:14:41.535539 30859 catalog_manager.cc:5495] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b reported cstate change: term changed from 0 to 1, leader changed from <none> to aa00712eca12447ba1d68c9aa5178d6b (127.8.238.2). New cstate: current_term: 1 leader_uuid: "aa00712eca12447ba1d68c9aa5178d6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa00712eca12447ba1d68c9aa5178d6b" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 39549 } health_report { overall_health: HEALTHY } } }
W20260811 10:14:41.565618 31630 tablet.cc:2369] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:14:41.587417 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.583949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.8.238.2:39549)
0811 10:14:41.584056 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:41.584058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:41.584067 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:14:41.584111 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:41.584112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:41.584137 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:41.584210 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:14:41.584414 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.585162 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.586284 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:41.586553 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:41.586621 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:41.586854 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:14:41.587002 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:14:41.587085 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:41.587315 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 10:14:41.587335 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 10:14:41.590137 31112 raft_consensus.cc:1261] T 8452e310e1ee481aab21cd02c9c4049e P d6676d98905b4532bceb403ee80e1776 [term 1 FOLLOWER]: Refusing update from remote peer aa00712eca12447ba1d68c9aa5178d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:41.590602 31706 consensus_queue.cc:1035] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6676d98905b4532bceb403ee80e1776" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:41.593075 31556 raft_consensus.cc:1261] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 1 FOLLOWER]: Refusing update from remote peer aa00712eca12447ba1d68c9aa5178d6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:41.593443 31706 consensus_queue.cc:1035] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "12bc75804d4d402fbd4c52639c53c5bf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:41.598133 31710 mvcc.cc:205] Tried to move back new op lower bound from 7317271681389780992 to 7317271681163640832. Current Snapshot: MvccSnapshot[applied={T|T < 7317271681389780992}]
I20260811 10:14:41.599011 31717 mvcc.cc:205] Tried to move back new op lower bound from 7317271681389780992 to 7317271681163640832. Current Snapshot: MvccSnapshot[applied={T|T < 7317271681389780992}]
I20260811 10:14:41.601110 31711 mvcc.cc:205] Tried to move back new op lower bound from 7317271681389780992 to 7317271681163640832. Current Snapshot: MvccSnapshot[applied={T|T < 7317271681389780992}]
I20260811 10:14:41.603756 30859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:41.603860 30859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:41.604693 30859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16d2616e33124edf921322180ad3f0bb: Sending DeleteTablet for 3 replicas of tablet 8452e310e1ee481aab21cd02c9c4049e
I20260811 10:14:41.605262 31522 tablet_service.cc:1514] Processing DeleteTablet for tablet 8452e310e1ee481aab21cd02c9c4049e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:41 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:41750
I20260811 10:14:41.605510 31310 tablet_service.cc:1514] Processing DeleteTablet for tablet 8452e310e1ee481aab21cd02c9c4049e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:41 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48004
I20260811 10:14:41.605741 31048 tablet_service.cc:1514] Processing DeleteTablet for tablet 8452e310e1ee481aab21cd02c9c4049e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:41 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:58250
I20260811 10:14:41.606483 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 30925
I20260811 10:14:41.617061 31729 tablet_replica.cc:330] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b: stopping tablet replica
I20260811 10:14:41.617254 31728 tablet_replica.cc:330] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: stopping tablet replica
I20260811 10:14:41.617452 31728 raft_consensus.cc:2229] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:41.617614 31729 raft_consensus.cc:2229] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:14:41.617784 31728 raft_consensus.cc:2258] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:41.617995 31729 raft_consensus.cc:2258] T 8452e310e1ee481aab21cd02c9c4049e P aa00712eca12447ba1d68c9aa5178d6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:41.619200 31728 ts_tablet_manager.cc:1905] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:14:41.621392 31728 ts_tablet_manager.cc:1918] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:41.621534 31728 log.cc:1196] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/8452e310e1ee481aab21cd02c9c4049e
I20260811 10:14:41.621909 31728 ts_tablet_manager.cc:1939] T 8452e310e1ee481aab21cd02c9c4049e P 12bc75804d4d402fbd4c52639c53c5bf: Deleting consensus metadata
I20260811 10:14:41.622426 30830 catalog_manager.cc:4841] TS 12bc75804d4d402fbd4c52639c53c5bf (127.8.238.3:34907): tablet 8452e310e1ee481aab21cd02c9c4049e (table test-table [id=a11251c15bde4da9b89c1bd18391577a]) successfully deleted
W20260811 10:14:41.624092 30837 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:34765 (error 108)
I20260811 10:14:41.624758 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 31198
I20260811 10:14:41.641999 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.641085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34765 (local address 127.0.0.1:58260)
0811 10:14:41.641873 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.641941 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":758}
W20260811 10:14:41.643026 30837 catalog_manager.cc:4584] TS d6676d98905b4532bceb403ee80e1776 (127.8.238.1:34765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8452e310e1ee481aab21cd02c9c4049e: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34765: connect: Connection refused (error 111)
I20260811 10:14:41.649329 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:41.649197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:39549 (local address 127.0.0.1:48016)
0811 10:14:41.649250 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:41.649292 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.2:39549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:14:41.649323 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 31418
I20260811 10:14:41.665042 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 30802
I20260811 10:14:41.679710 9144 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 10:14:42.024719 9144 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 10:14:42.024827 9144 mini_ranger.cc:66] Stopping Ranger...
I20260811 10:14:42.379243 9144 mini_ranger.cc:68] Stopped Ranger
2026-08-11 10:14:42.379 UTC [29250] LOG: received smart shutdown request
2026-08-11 10:14:42.383 UTC [29250] LOG: background worker "logical replication launcher" (PID 29270) exited with exit code 1
2026-08-11 10:14:42.384 UTC [29256] LOG: shutting down
2026-08-11 10:14:42.554 UTC [29250] LOG: database system is shut down
2026-08-11T10:14:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48434 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:14:42 dist-test-slave-7f01 krb5kdc[31745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:14:42 dist-test-slave-7f01 krb5kdc[31745](info): set up 2 sockets
Aug 11 10:14:42 dist-test-slave-7f01 krb5kdc[31745](info): commencing operation
krb5kdc: starting...
W20260811 10:14:43.963692 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:14:44.096601 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34019
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:44.180959 31770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:44.181200 31770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:44.181241 31770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:44.182364 31770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:44.182405 31770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:44.182420 31770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:44.182432 31770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:44.182446 31770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:44.183979 31770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:44.184212 31770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:44.184468 31770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:44.189296 31786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:44.189361 31779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:44.189527 31770 server_base.cc:1034] running on GCE node
W20260811 10:14:44.189371 31782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:44.189829 31770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:44.190155 31770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:44.191306 31770 hybrid_clock.cc:648] HybridClock initialized: now 1786443284191281 us; error 44 us; skew 500 ppm
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:44.196812 31770 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:44.197302 31770 webserver.cc:460] Webserver started at http://127.8.238.62:46577/ using document root <none> and password file <none>
I20260811 10:14:44.197549 31770 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:44.197619 31770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:44.197764 31770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:44.198810 31770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "b6229458467e4ceba6b0d639292635e0"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "b0e0485d62234a1c339b6d6be35600f5"
server_key_iv: "a06c1af730caed13467592ba6076ad83"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.199184 31770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "b6229458467e4ceba6b0d639292635e0"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "b0e0485d62234a1c339b6d6be35600f5"
server_key_iv: "a06c1af730caed13467592ba6076ad83"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.200708 31770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:14:44.201481 31799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:44.201673 31770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:14:44.201782 31770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "b6229458467e4ceba6b0d639292635e0"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "b0e0485d62234a1c339b6d6be35600f5"
server_key_iv: "a06c1af730caed13467592ba6076ad83"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.201864 31770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:44.210721 31770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:44.212785 31770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:44.212956 31770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:44.217275 31770 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:34019
I20260811 10:14:44.217306 31887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:34019 every 8 connection(s)
I20260811 10:14:44.217643 31770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:44.218467 31888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:44.220062 31888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.221851 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 31770
I20260811 10:14:44.221949 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:14:44.222159 9144 external_mini_cluster.cc:1428] Setting key 9aca62774809603619b14741c97c2adf
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:44.230146 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.220924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.8.238.62:34019)
0811 10:14:44.221040 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.221045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.221077 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:14:44.221595 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.221597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.221612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.221707 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.222389 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.223192 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.224160 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.224465 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.224590 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.226956 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.226963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.226964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.226971 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.228577 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.229057 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.229060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.229060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.229139 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.229387 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.229388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.229388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.229577 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.229638 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.229900 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.229995 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":43,"threads_started":1}
I20260811 10:14:44.230247 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.220760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34019 (local address 127.0.0.1:45024)
0811 10:14:44.220916 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.220947 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.221432 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.221713 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.221716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.222091 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.222290 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.222295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.223301 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.223302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.224063 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.224065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.224564 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.224565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.224631 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.225145 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.225153 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.226816 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.228670 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.228678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.228681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.228975 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.229207 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.229208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.229209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.229322 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.229649 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.229657 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.229742 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.229859 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.230105 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":77,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.8.238.61@KRBTEST.COM; defaulting to no policy
I20260811 10:14:44.231266 31888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.231688 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.231551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41568)
0811 10:14:44.231608 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.231662 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:14:44.231853 31803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
I20260811 10:14:44.232298 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.232195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41574)
0811 10:14:44.232246 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.232270 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 10:14:44.232402 31888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33055: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
I20260811 10:14:44.250236 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:33055
--webserver_interface=127.8.238.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:44.281632 31888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } attempt: 1
I20260811 10:14:44.282193 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.282049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41586)
0811 10:14:44.282121 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.282162 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 10:14:44.282645 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.282555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41596)
0811 10:14:44.282600 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.282627 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 10:14:44.282759 31888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33055: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
W20260811 10:14:44.334687 31902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:44.334949 31902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:44.334992 31902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:44.336213 31902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:44.336284 31902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:44.336308 31902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:44.336345 31902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:44.336376 31902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:44.337873 31902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.238.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:44.338152 31902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:44.338387 31902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:44.343391 31917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:44.343348 31914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:44.343357 31913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:44.343606 31902 server_base.cc:1034] running on GCE node
I20260811 10:14:44.343899 31902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:44.344178 31902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:44.345316 31902 hybrid_clock.cc:648] HybridClock initialized: now 1786443284345289 us; error 49 us; skew 500 ppm
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:44.347913 31902 init.cc:377] Logged in from keytab as kudu/127.8.238.61@KRBTEST.COM (short username kudu)
I20260811 10:14:44.348398 31902 webserver.cc:460] Webserver started at http://127.8.238.61:45617/ using document root <none> and password file <none>
I20260811 10:14:44.348642 31902 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:44.348712 31902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:44.348865 31902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:44.349896 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data/instance:
uuid: "0400ef4dd8b84f359df7d1d6af7d4968"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "584734f71ee5111843c1cae7705c1c7b"
server_key_iv: "5b411f5e72a4ae3fb4599e8d8f89defb"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.350263 31902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal/instance:
uuid: "0400ef4dd8b84f359df7d1d6af7d4968"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "584734f71ee5111843c1cae7705c1c7b"
server_key_iv: "5b411f5e72a4ae3fb4599e8d8f89defb"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.350924 31888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } attempt: 2
I20260811 10:14:44.351372 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.351213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41598)
0811 10:14:44.351288 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.351330 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 10:14:44.351821 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.351678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41610)
0811 10:14:44.351728 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.351795 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:14:44.351876 31902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
W20260811 10:14:44.351939 31888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33055: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33055: connect: Connection refused (error 111)
I20260811 10:14:44.352735 31925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:44.352936 31902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:44.353029 31902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "0400ef4dd8b84f359df7d1d6af7d4968"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "584734f71ee5111843c1cae7705c1c7b"
server_key_iv: "5b411f5e72a4ae3fb4599e8d8f89defb"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.353094 31902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:44.380759 31902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:44.382822 31902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:44.382949 31902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:44.387538 31902 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.61:33055
I20260811 10:14:44.387614 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.61:33055 every 8 connection(s)
I20260811 10:14:44.387997 31902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
I20260811 10:14:44.388962 32019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:44.390555 32019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.395437 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 31902
I20260811 10:14:44.395574 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-1/wal/instance
I20260811 10:14:44.395799 9144 external_mini_cluster.cc:1428] Setting key 726d1edd34cf3b3269ebe0cd5a763651
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:44.400413 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.391247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.8.238.62:34019)
0811 10:14:44.391349 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.391353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.391518 (+ 165us) server_negotiation.cc:407] Connection header received
0811 10:14:44.392132 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.392133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.392149 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.392205 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.392837 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.393349 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.394340 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.394560 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.394642 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.397303 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.397305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.397306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.397314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.398915 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.399322 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.399324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.399324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.399410 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.399649 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.399650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.399650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.399837 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.399910 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.400173 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.400273 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 10:14:44.400594 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.391194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34019 (local address 127.0.0.1:45028)
0811 10:14:44.391397 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.391432 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.392010 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.392281 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.392285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.392545 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.392738 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.392742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.393451 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.393452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.394238 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.394240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.394647 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.394648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.394775 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.395259 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.395277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.397204 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.399007 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.399014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.399015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.399243 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.399478 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.399479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.399479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.399581 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.399911 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.399916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.400037 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.400123 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.400313 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260811 10:14:44.401190 32019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.60@KRBTEST.COM" created.
I20260811 10:14:44.409463 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.401488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41626)
0811 10:14:44.401537 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.401548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.401590 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.401877 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.401878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.402007 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.402058 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.402060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.403394 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.403395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.403938 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.403940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.404588 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.404589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.404648 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.405047 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.405051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.406561 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.408315 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.408316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.408316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.408484 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.408763 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.408763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.408764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.408820 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.409099 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.409100 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.409138 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.409218 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.409357 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 10:14:44.409647 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.401541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.8.238.61:33055)
0811 10:14:44.401703 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.401708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.401733 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:14:44.401759 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.401761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.401777 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.401873 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.402135 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.403300 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.404016 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.404510 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.404591 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.406642 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.406657 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.406657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.406663 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.408240 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.408593 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.408596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.408597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.408682 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.408884 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.408885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.408885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.409024 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.409089 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.409260 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.409458 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260811 10:14:44.410245 32019 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.410614 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.410474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34778)
0811 10:14:44.410528 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.410580 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:14:44.410768 31929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
I20260811 10:14:44.411274 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.411165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34782)
0811 10:14:44.411213 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.411244 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 10:14:44.411430 32019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:38753: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
I20260811 10:14:44.426592 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:38753
--webserver_interface=127.8.238.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:44.443099 31888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } attempt: 3
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:14:44.450690 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.443479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41628)
0811 10:14:44.443550 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.443559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.443626 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.443839 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.443841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.444012 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.444086 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.444088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.444796 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.444798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.445325 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.445327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.445746 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.445748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.445837 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.446329 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.446334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.447843 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.449546 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.449548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.449548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.449746 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.449989 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.449990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.449990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.450055 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.450323 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.450324 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.450373 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.450477 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.450586 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:14:44.450690 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.443563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.8.238.61:33055)
0811 10:14:44.443637 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.443639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.443652 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:14:44.443685 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.443686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.443697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.443826 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.444177 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.444697 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.445423 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.445663 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.445748 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.447932 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.447934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.447934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.447942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.449448 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.449839 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.449840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.449841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.449903 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.450133 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.450133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.450134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.450251 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.450331 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.450505 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.450588 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:14:44.451313 31888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.451690 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.451580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34790)
0811 10:14:44.451635 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.451667 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 10:14:44.451994 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.451931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34796)
0811 10:14:44.451956 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.451980 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260811 10:14:44.452098 31888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:38753: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
I20260811 10:14:44.460628 32019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } attempt: 1
I20260811 10:14:44.460995 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.460877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34806)
0811 10:14:44.460931 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.460968 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:14:44.461381 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.461294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34816)
0811 10:14:44.461339 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.461364 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:14:44.461486 32019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:38753: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
I20260811 10:14:44.483245 31888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } attempt: 1
I20260811 10:14:44.483723 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.483584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34824)
0811 10:14:44.483652 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.483690 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 10:14:44.484166 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.484057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34832)
0811 10:14:44.484115 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.484139 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 10:14:44.484256 31888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:38753: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
W20260811 10:14:44.512729 32028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:44.512956 32028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:44.512979 32028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:44.514166 32028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:44.514211 32028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:44.514225 32028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:44.514238 32028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:44.514297 32028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:44.515892 32028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:34019,127.8.238.61:33055,127.8.238.60:38753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:38753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.238.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:44.516140 32028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:44.516383 32028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:44.521242 32035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:44.521253 32036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:44.521245 32042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:44.521533 32028 server_base.cc:1034] running on GCE node
I20260811 10:14:44.521804 32028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:44.522063 32028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:44.523218 32028 hybrid_clock.cc:648] HybridClock initialized: now 1786443284523207 us; error 30 us; skew 500 ppm
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:44.526000 32028 init.cc:377] Logged in from keytab as kudu/127.8.238.60@KRBTEST.COM (short username kudu)
I20260811 10:14:44.526443 32028 webserver.cc:460] Webserver started at http://127.8.238.60:40951/ using document root <none> and password file <none>
I20260811 10:14:44.526665 32028 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:44.526712 32028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:44.526942 32028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:44.528132 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data/instance:
uuid: "110b960492af4e98b20f656847557583"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "0929b0b7547ba856d8d7394fcb3043b0"
server_key_iv: "1cc2c06e11e618a4e10c8b059ffed6e8"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.528589 32028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal/instance:
uuid: "110b960492af4e98b20f656847557583"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "0929b0b7547ba856d8d7394fcb3043b0"
server_key_iv: "1cc2c06e11e618a4e10c8b059ffed6e8"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.529626 32019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } attempt: 2
I20260811 10:14:44.530045 32028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:14:44.530064 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.529929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34834)
0811 10:14:44.529992 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.530022 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:14:44.530416 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.530319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34840)
0811 10:14:44.530365 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.530391 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 10:14:44.530514 32019 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:38753: Network error: Client connection negotiation failed: client connection to 127.8.238.60:38753: connect: Connection refused (error 111)
I20260811 10:14:44.530887 32055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:44.531091 32028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:44.531188 32028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "110b960492af4e98b20f656847557583"
format_stamp: "Formatted at 2026-08-11 10:14:44 on dist-test-slave-7f01"
server_key: "0929b0b7547ba856d8d7394fcb3043b0"
server_key_iv: "1cc2c06e11e618a4e10c8b059ffed6e8"
server_key_version: "encryptionkey@0"
I20260811 10:14:44.531261 32028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:44.540923 32028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:44.542713 32028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:44.542862 32028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:44.546991 32028 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.60:38753
I20260811 10:14:44.547067 32142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.60:38753 every 8 connection(s)
I20260811 10:14:44.547413 32028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
I20260811 10:14:44.548341 32144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:44.549901 32144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.552508 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 32028
I20260811 10:14:44.552626 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786443124069174-9144-0/minicluster-data/master-2/wal/instance
I20260811 10:14:44.552850 9144 external_mini_cluster.cc:1428] Setting key 23039a9d7e51827cf2fd1365e11a699a
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:14:44.559404 31888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } attempt: 2
I20260811 10:14:44.560362 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.550583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.8.238.62:34019)
0811 10:14:44.550645 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.550651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.550823 (+ 172us) server_negotiation.cc:407] Connection header received
0811 10:14:44.551282 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.551283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.551292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.551350 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.552000 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.552590 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.554167 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.554484 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.554570 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.556994 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.556996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.556996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.557002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.558714 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.559163 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.559165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.559165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.559232 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.559531 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.559532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.559533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.559686 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.559770 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.560135 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.560261 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:14:44.560495 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.550524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:34019 (local address 127.0.0.1:45034)
0811 10:14:44.550695 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.550734 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.551197 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.551421 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.551424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.551660 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.551917 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.551922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.552731 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.552732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.554054 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.554058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.554583 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.554584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.554695 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.555208 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.555216 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.556902 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.558816 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.558824 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.558827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.559074 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.559309 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.559310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.559311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.559439 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.559809 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.559817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.559984 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.560095 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.560262 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260811 10:14:44.561086 32144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.562817 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.553240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.8.238.60:38753)
0811 10:14:44.553404 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.553407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.553441 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:14:44.553513 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.553514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.553530 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.553591 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.554330 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.555314 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.556294 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.556820 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.556876 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.559215 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.559224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.559225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.559231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.561559 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.561970 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.561973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.561974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.562032 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.562246 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.562247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.562248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.562391 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.562573 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.562616 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.562688 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:44.568907 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.559666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34862)
0811 10:14:44.559716 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.559728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.559793 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.560120 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.560121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.560793 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.560844 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.560847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.561781 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.561782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.562519 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.562521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.562963 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.562964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.563035 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.563434 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.563439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.564990 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.567686 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.567687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.567688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.567942 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.568187 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.568188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.568262 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.568595 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.568596 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.568645 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.568707 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.568807 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:14:44.568957 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.559799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.8.238.60:38753)
0811 10:14:44.559960 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.559962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.559969 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:14:44.559991 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.559992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.560003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.560924 (+ 921us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.560956 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.561703 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.562609 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.562881 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.563184 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.565076 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.565077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.565078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.565085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.567572 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568037 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.568039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.568039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.568102 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568358 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.568359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.568359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.568519 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.568586 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.568745 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.568818 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260811 10:14:44.569806 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.561332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33055 (local address 127.0.0.1:41642)
0811 10:14:44.561447 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.561456 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.561518 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.561698 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.561699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.561844 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.561902 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.561903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.562766 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.562767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.563716 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.563720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.564209 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.564210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.564294 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.565215 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.565220 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.566665 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.568375 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.568376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.568588 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.568843 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.568844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.568924 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.569516 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.569517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.569563 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.569638 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.569732 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 10:14:44.569865 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.561397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.8.238.61:33055)
0811 10:14:44.561451 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.561453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.561506 (+ 53us) server_negotiation.cc:407] Connection header received
0811 10:14:44.561582 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.561583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.561593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.561643 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.561986 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.562691 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.563820 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.564128 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.564200 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.566759 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.566761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.566761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.566770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.568274 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.568698 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.568700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.568700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.568760 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.569308 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.569309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.569310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.569460 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.569524 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.569690 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.569772 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:14:44.570886 31888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: Bootstrap starting.
I20260811 10:14:44.571861 32144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:14:44.572109 31888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:44.572495 31888 log.cc:824] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:44.573851 31888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: No bootstrap required, opened a new log
I20260811 10:14:44.574079 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.563810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.8.238.62:34019)
0811 10:14:44.563847 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.563848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.563855 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:14:44.563873 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.563873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.563881 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.563933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.564571 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.565072 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.566051 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.566369 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.566443 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.569224 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.569225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.569225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.569233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.571108 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.572167 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.572168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.572168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.572247 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.573384 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.573386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.573386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.573554 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.573714 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.573753 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.573821 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:14:44.576277 31888 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.576459 31888 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:44.576499 31888 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6229458467e4ceba6b0d639292635e0, State: Initialized, Role: FOLLOWER
I20260811 10:14:44.576627 31888 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.576924 32159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.576982 31888 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:44.577039 32159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:14:44.579064 32173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:14:44.579119 32173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:14:44.579679 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.572429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34864)
0811 10:14:44.572555 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.572564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.572611 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.572730 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.572731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.572879 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.572913 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.572914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.573753 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.573754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.574450 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.574453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.574916 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.574918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.574991 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.575409 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.575415 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.576624 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.578514 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.578515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.578515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.578729 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.578958 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.578959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.578959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.579014 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.579382 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.579383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.579426 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.579495 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.579580 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:14:44.580248 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.572513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.8.238.60:38753)
0811 10:14:44.572551 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.572553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.572598 (+ 45us) server_negotiation.cc:407] Connection header received
0811 10:14:44.572659 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.572660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.572670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.572729 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.572980 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.573645 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.574530 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.574851 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.574919 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.576729 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.576730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.576731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.576737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.578425 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.578815 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.578816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.578816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.578890 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.579070 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.579070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.579071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.579230 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.579289 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.580096 (+ 807us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.580176 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:14:44.579819 31770 master_runner.cc:186] Error getting master registration for 127.8.238.61:33055: OK, instance_id { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" instance_seqno: 1786443284386175 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:14:44.582316 31770 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:14:44.583904 32144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: Bootstrap starting.
I20260811 10:14:44.584686 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.574674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.8.238.61:33055)
0811 10:14:44.574714 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.574716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.574724 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:14:44.574794 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.574795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.574805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.574851 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.575465 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.576170 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.577403 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.577723 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.577791 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.580501 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.580503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.580503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.580510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.582235 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.583817 (+ 1582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.583818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.583818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.583881 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.584109 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.584110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.584110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.584248 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.584470 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.584510 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.584588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:14:44.584714 32144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:44.585057 32144 log.cc:824] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:44.585951 32144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: No bootstrap required, opened a new log
I20260811 10:14:44.587018 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:14:44.588157 32144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.588320 32144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:44.588358 32144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110b960492af4e98b20f656847557583, State: Initialized, Role: FOLLOWER
I20260811 10:14:44.588495 32144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.588762 32180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.588814 32144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:44.588881 32180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:14:44.590941 32197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:14:44.591001 32197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:14:44.592555 32028 master_runner.cc:186] Error getting master registration for 127.8.238.61:33055: OK, instance_id { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" instance_seqno: 1786443284386175 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:14:44.594863 32028 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:14:44.604663 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.592024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.8.238.62:34019)
0811 10:14:44.592067 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.592069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.595937 (+ 3868us) server_negotiation.cc:407] Connection header received
0811 10:14:44.595989 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.595990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.596001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.596078 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.596936 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.597719 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.598802 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.599089 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.599132 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.600881 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.600882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.600883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.600891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.602606 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.603730 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.603731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.603731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.603823 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.604036 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.604037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.604037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.604180 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.604254 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.604486 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.604564 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:14:44.605048 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.592789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.8.238.60:38753)
0811 10:14:44.592824 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.592826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.596384 (+ 3558us) server_negotiation.cc:407] Connection header received
0811 10:14:44.596483 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.596484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.596494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.596523 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.597162 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.597894 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.598924 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.599215 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.599254 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.601689 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.601691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.601691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.601698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.603352 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.604151 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.604152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.604153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.604209 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.604396 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.604397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.604397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.604529 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.604624 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.604876 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.604959 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:14:44.610384 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.592262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.8.238.61:33055)
0811 10:14:44.592309 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.592311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.596072 (+ 3761us) server_negotiation.cc:407] Connection header received
0811 10:14:44.596163 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.596164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.596173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.596201 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.600086 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.600795 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.603249 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.603567 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.603607 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.607374 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.607376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.607376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.607385 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.609114 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.609526 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.609527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.609527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.609595 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.609811 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.609812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.609812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.609956 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.610053 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.610199 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.610284 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:14:44.621662 32019 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } attempt: 3
Aug 11 10:14:44 dist-test-slave-7f01 krb5kdc[31745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:14:44.630234 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.621954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:38753 (local address 127.0.0.1:34886)
0811 10:14:44.622005 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:44.622016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:44.622053 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:44.622192 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:44.622194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:44.622398 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:44.622451 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.622453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.623377 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.623379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.624264 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:44.624267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.624842 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.624844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:44.624908 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.625407 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:44.625413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:44.627053 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:44.629040 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.629041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.629042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.629298 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.629529 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:44.629530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:44.629531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:44.629603 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:44.629900 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:44.629901 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:44.629952 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 10:14:44.630026 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:14:44.630130 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 10:14:44.630250 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:44.622017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.8.238.60:38753)
0811 10:14:44.622060 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:44.622061 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:44.622073 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:14:44.622098 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:44.622099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:44.622109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:44.622174 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:44.622534 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.623287 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.624360 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:44.624678 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:44.624772 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:44.627148 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:44.627149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:44.627150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:44.627158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:44.628944 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.629379 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.629381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.629381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.629446 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:44.629675 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:44.629675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:44.629676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:44.629826 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:44.629895 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:44.630066 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 10:14:44.630150 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:14:44.632407 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: Bootstrap starting.
I20260811 10:14:44.633319 32019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:44.633702 32019 log.cc:824] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:44.636533 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: No bootstrap required, opened a new log
I20260811 10:14:44.638620 32019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.638767 32019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:44.638803 32019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0400ef4dd8b84f359df7d1d6af7d4968, State: Initialized, Role: FOLLOWER
I20260811 10:14:44.638947 32019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.639278 32019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 10:14:44.641255 32227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:14:44.641372 32227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:14:44.639238 32209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.641690 32209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:14:44.643612 31902 master_runner.cc:263] No leader master found from master 0400ef4dd8b84f359df7d1d6af7d4968
I20260811 10:14:44.643669 31902 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 10:14:44.713738 31807 tablet.cc:2369] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:14:44.744349 32159 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:44.744508 32159 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.744782 32159 leader_election.cc:288] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0400ef4dd8b84f359df7d1d6af7d4968 (127.8.238.61:33055), 110b960492af4e98b20f656847557583 (127.8.238.60:38753)
I20260811 10:14:44.745087 31972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6229458467e4ceba6b0d639292635e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" is_pre_election: true
I20260811 10:14:44.745082 32108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6229458467e4ceba6b0d639292635e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110b960492af4e98b20f656847557583" is_pre_election: true
I20260811 10:14:44.745235 31972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6229458467e4ceba6b0d639292635e0 in term 0.
I20260811 10:14:44.745235 32108 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6229458467e4ceba6b0d639292635e0 in term 0.
I20260811 10:14:44.745524 31803 leader_election.cc:302] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0400ef4dd8b84f359df7d1d6af7d4968, b6229458467e4ceba6b0d639292635e0; no voters:
I20260811 10:14:44.745679 32159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:14:44.745733 32159 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:14:44.745767 32159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:44.746582 32159 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.746726 32159 leader_election.cc:288] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [CANDIDATE]: Term 1 election: Requested vote from peers 0400ef4dd8b84f359df7d1d6af7d4968 (127.8.238.61:33055), 110b960492af4e98b20f656847557583 (127.8.238.60:38753)
I20260811 10:14:44.746958 31972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6229458467e4ceba6b0d639292635e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0400ef4dd8b84f359df7d1d6af7d4968"
I20260811 10:14:44.746973 32108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6229458467e4ceba6b0d639292635e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110b960492af4e98b20f656847557583"
I20260811 10:14:44.747042 31972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:44.747056 32108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:44.747949 31972 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6229458467e4ceba6b0d639292635e0 in term 1.
I20260811 10:14:44.747941 32108 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6229458467e4ceba6b0d639292635e0 in term 1.
I20260811 10:14:44.748194 31803 leader_election.cc:302] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0400ef4dd8b84f359df7d1d6af7d4968, b6229458467e4ceba6b0d639292635e0; no voters:
I20260811 10:14:44.748322 32159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:44.748441 32159 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [term 1 LEADER]: Becoming Leader. State: Replica: b6229458467e4ceba6b0d639292635e0, State: Running, Role: LEADER
I20260811 10:14:44.748540 32159 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } }
I20260811 10:14:44.748970 32229 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6229458467e4ceba6b0d639292635e0. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.749119 32229 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:44.749390 32232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:44.749656 32232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:44.751164 32108 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [term 1 FOLLOWER]: Refusing update from remote peer b6229458467e4ceba6b0d639292635e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:44.751430 31972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [term 1 FOLLOWER]: Refusing update from remote peer b6229458467e4ceba6b0d639292635e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:44.751631 32229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:44.751804 32229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:44.752972 32209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6229458467e4ceba6b0d639292635e0. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.753078 32209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:14:44.753150 32180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6229458467e4ceba6b0d639292635e0. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.753247 32180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:14:44.754263 32229 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.754354 32229 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:44.754437 32180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.754510 32180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:14:44.754642 32233 mvcc.cc:205] Tried to move back new op lower bound from 7317271694338813952 to 7317271694330564608. Current Snapshot: MvccSnapshot[applied={T|T < 7317271694338813952}]
I20260811 10:14:44.754760 32236 mvcc.cc:205] Tried to move back new op lower bound from 7317271694338813952 to 7317271694330564608. Current Snapshot: MvccSnapshot[applied={T|T < 7317271694338813952}]
I20260811 10:14:44.755648 32159 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.755769 32159 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:44.755990 32209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6229458467e4ceba6b0d639292635e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6229458467e4ceba6b0d639292635e0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34019 } } peers { permanent_uuid: "0400ef4dd8b84f359df7d1d6af7d4968" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33055 } } peers { permanent_uuid: "110b960492af4e98b20f656847557583" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 38753 } } }
I20260811 10:14:44.756067 32209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:14:44.756188 32234 mvcc.cc:205] Tried to move back new op lower bound from 7317271694338813952 to 7317271694330564608. Current Snapshot: MvccSnapshot[applied={T|T < 7317271694338813952}]
I20260811 10:14:44.756965 32232 catalog_manager.cc:1347] Generated new cluster ID: 910e7542bcf849febd99334d10a6f12a
I20260811 10:14:44.757032 32232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:44.770102 32232 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:14:44.770819 32232 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:44.792368 32232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6229458467e4ceba6b0d639292635e0: Generated new TSK 0
I20260811 10:14:44.792582 32232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:14:45.591496 32197 catalog_manager.cc:1259] Loaded cluster ID: 910e7542bcf849febd99334d10a6f12a
I20260811 10:14:45.591576 32197 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: loading cluster ID for follower catalog manager: success
I20260811 10:14:45.592447 32197 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: acquiring CA information for follower catalog manager: success
I20260811 10:14:45.592722 32197 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 110b960492af4e98b20f656847557583: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:14:45.641881 32227 catalog_manager.cc:1259] Loaded cluster ID: 910e7542bcf849febd99334d10a6f12a
I20260811 10:14:45.641975 32227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: loading cluster ID for follower catalog manager: success
I20260811 10:14:45.642838 32227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: acquiring CA information for follower catalog manager: success
I20260811 10:14:45.643158 32227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0400ef4dd8b84f359df7d1d6af7d4968: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:14:45.850840 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 31770
I20260811 10:14:45.864120 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 31902
I20260811 10:14:45.878527 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 32028
2026-08-11T10:14:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3206 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:14:45 dist-test-slave-7f01 krb5kdc[32255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:14:45 dist-test-slave-7f01 krb5kdc[32255](info): set up 2 sockets
Aug 11 10:14:45 dist-test-slave-7f01 krb5kdc[32255](info): commencing operation
krb5kdc: starting...
W20260811 10:14:47.163270 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:14:47.286195 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:36977
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:36977
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:47.369102 32272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:47.369313 32272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:47.369338 32272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:47.370417 32272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:47.370451 32272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:47.370465 32272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:47.370478 32272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:47.370491 32272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:47.371981 32272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:36977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:36977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:47.372179 32272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:47.372368 32272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:47.377168 32279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:47.377210 32278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:47.377166 32283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:47.377579 32272 server_base.cc:1034] running on GCE node
I20260811 10:14:47.377840 32272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:47.378126 32272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:47.379292 32272 hybrid_clock.cc:648] HybridClock initialized: now 1786443287379284 us; error 32 us; skew 500 ppm
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443287, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:47.381958 32272 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:47.382372 32272 webserver.cc:460] Webserver started at http://127.8.238.62:33003/ using document root <none> and password file <none>
I20260811 10:14:47.382606 32272 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:47.382673 32272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:47.382825 32272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:47.383931 32272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "fb84318841e34b7bb35c0ab38dd097df"
format_stamp: "Formatted at 2026-08-11 10:14:47 on dist-test-slave-7f01"
server_key: "0a6aceda315a5b19d229508d7701ec95"
server_key_iv: "a184c79a904d3168d25e21966a23db49"
server_key_version: "encryptionkey@0"
I20260811 10:14:47.384287 32272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "fb84318841e34b7bb35c0ab38dd097df"
format_stamp: "Formatted at 2026-08-11 10:14:47 on dist-test-slave-7f01"
server_key: "0a6aceda315a5b19d229508d7701ec95"
server_key_iv: "a184c79a904d3168d25e21966a23db49"
server_key_version: "encryptionkey@0"
I20260811 10:14:47.385603 32272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:14:47.386291 32296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:47.386459 32272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:47.386560 32272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "fb84318841e34b7bb35c0ab38dd097df"
format_stamp: "Formatted at 2026-08-11 10:14:47 on dist-test-slave-7f01"
server_key: "0a6aceda315a5b19d229508d7701ec95"
server_key_iv: "a184c79a904d3168d25e21966a23db49"
server_key_version: "encryptionkey@0"
I20260811 10:14:47.386636 32272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:47.393402 32272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:47.395629 32272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:47.395812 32272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:47.399984 32272 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:36977
I20260811 10:14:47.400017 32378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:36977 every 8 connection(s)
I20260811 10:14:47.400380 32272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:47.400837 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 32272
I20260811 10:14:47.400944 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:14:47.401147 9144 external_mini_cluster.cc:1428] Setting key 2040e4f01b707133f8037aa75d2bc6bf
I20260811 10:14:47.401458 32379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:47.404984 32379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df: Bootstrap starting.
I20260811 10:14:47.405689 32379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:47.406055 32379 log.cc:824] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:47.406855 32379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df: No bootstrap required, opened a new log
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:47.409317 32379 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } }
I20260811 10:14:47.409480 32379 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:47.409533 32379 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb84318841e34b7bb35c0ab38dd097df, State: Initialized, Role: FOLLOWER
I20260811 10:14:47.409664 32379 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } }
I20260811 10:14:47.409758 32379 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:47.409804 32379 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:47.409853 32379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:47.410393 32379 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } }
I20260811 10:14:47.410521 32379 leader_election.cc:302] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb84318841e34b7bb35c0ab38dd097df; no voters:
I20260811 10:14:47.410704 32379 leader_election.cc:288] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:14:47.410753 32387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:47.410920 32387 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [term 1 LEADER]: Becoming Leader. State: Replica: fb84318841e34b7bb35c0ab38dd097df, State: Running, Role: LEADER
I20260811 10:14:47.410988 32379 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:47.411019 32387 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } }
I20260811 10:14:47.411360 32387 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb84318841e34b7bb35c0ab38dd097df. Latest consensus state: current_term: 1 leader_uuid: "fb84318841e34b7bb35c0ab38dd097df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } } }
I20260811 10:14:47.411456 32387 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:47.411370 32390 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb84318841e34b7bb35c0ab38dd097df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb84318841e34b7bb35c0ab38dd097df" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 36977 } } }
I20260811 10:14:47.411624 32390 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:47.411952 32400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:47.412599 32400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:47.413007 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:47.401705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.8.238.62:36977)
0811 10:14:47.401912 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:47.401921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:47.401957 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:14:47.402626 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:47.402636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:47.402699 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:47.402875 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:47.403467 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:47.404884 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:47.405642 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:47.406280 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:47.406406 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:47.409126 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:47.409130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:47.409131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:47.409141 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:47.411088 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:47.411540 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:47.411543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:47.411544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:47.411649 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:47.411932 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:47.411932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:47.411933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:47.412235 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:47.412387 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:47.412662 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 10:14:47.412806 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260811 10:14:47.414099 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:14:47.414384 32400 catalog_manager.cc:1347] Generated new cluster ID: 1b81bc9103c645e09c852dec407eb098
I20260811 10:14:47.414444 32400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:47.436264 32400 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:14:47.437014 32400 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:47.444905 32400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb84318841e34b7bb35c0ab38dd097df: Generated new TSK 0
I20260811 10:14:47.445102 32400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:14:47.468144 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 32272
2026-08-11T10:14:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1590 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32422](info): set up 2 sockets
Aug 11 10:14:47 dist-test-slave-7f01 krb5kdc[32422](info): commencing operation
krb5kdc: starting...
W20260811 10:14:48.778868 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:14:48 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:14:48.902496 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:48.986145 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:48.986369 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:48.986394 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:48.987499 32438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:48.987536 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:48.987550 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:48.987562 32438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:48.987574 32438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:48.989017 32438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:48.989207 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:48.989373 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:48.994202 32448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:48.994196 32445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:48.994222 32444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:48.994472 32438 server_base.cc:1034] running on GCE node
I20260811 10:14:48.994726 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:48.994978 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:48.996158 32438 hybrid_clock.cc:648] HybridClock initialized: now 1786443288996140 us; error 37 us; skew 500 ppm
Aug 11 10:14:48 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443288, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:48.998734 32438 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:48.999164 32438 webserver.cc:460] Webserver started at http://127.8.238.62:35565/ using document root <none> and password file <none>
I20260811 10:14:48.999404 32438 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:48.999452 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:48.999609 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:49.000691 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "e1fcc1d4d03e4264a72515238fb05265"
format_stamp: "Formatted at 2026-08-11 10:14:48 on dist-test-slave-7f01"
server_key: "eeb89f362d4ad7ac2455c6e94a9c7627"
server_key_iv: "74be3b618f57feb84465f15d49dc9305"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.001041 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "e1fcc1d4d03e4264a72515238fb05265"
format_stamp: "Formatted at 2026-08-11 10:14:48 on dist-test-slave-7f01"
server_key: "eeb89f362d4ad7ac2455c6e94a9c7627"
server_key_iv: "74be3b618f57feb84465f15d49dc9305"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.002349 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:14:49.003050 32462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.003245 32438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:49.003348 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e1fcc1d4d03e4264a72515238fb05265"
format_stamp: "Formatted at 2026-08-11 10:14:48 on dist-test-slave-7f01"
server_key: "eeb89f362d4ad7ac2455c6e94a9c7627"
server_key_iv: "74be3b618f57feb84465f15d49dc9305"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.003413 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.026553 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.028903 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.029055 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.033231 32438 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39947
I20260811 10:14:49.033282 32544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39947 every 8 connection(s)
I20260811 10:14:49.033625 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:49.034580 32545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:49.036980 32545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap starting.
I20260811 10:14:49.037568 32545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:49.037869 32545 log.cc:824] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:49.038108 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 32438
I20260811 10:14:49.038220 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:14:49.038439 9144 external_mini_cluster.cc:1428] Setting key c492b51c0760fd860e7fecc360b65c0d
I20260811 10:14:49.038872 32545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: No bootstrap required, opened a new log
I20260811 10:14:49.041280 32545 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.041440 32545 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:49.041498 32545 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Initialized, Role: FOLLOWER
I20260811 10:14:49.041668 32545 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.041764 32545 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:49.041810 32545 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:49.041867 32545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:49.042404 32545 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.042524 32545 leader_election.cc:302] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1fcc1d4d03e4264a72515238fb05265; no voters:
I20260811 10:14:49.042698 32545 leader_election.cc:288] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:14:49.042735 32555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:49.042922 32555 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 LEADER]: Becoming Leader. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Running, Role: LEADER
I20260811 10:14:49.042985 32545 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:49.043066 32555 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.043402 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1fcc1d4d03e4264a72515238fb05265. Latest consensus state: current_term: 1 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:49.043416 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:49.043504 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:49.043538 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:49.044584 32578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:49.045084 32578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:49.046708 32578 catalog_manager.cc:1347] Generated new cluster ID: 8a60ef7cae434cfb8dc9a9380d9f1fa7
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:49.046772 32578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:49.051062 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.038938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.8.238.62:39947)
0811 10:14:49.039117 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.039125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.039151 (+ 26us) server_negotiation.cc:407] Connection header received
0811 10:14:49.040197 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.040204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.040253 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.040367 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:49.041180 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.042462 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.043536 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.044170 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.044268 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.047344 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:49.047348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:49.047348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:49.047359 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:49.049325 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.049751 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.049754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.049755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.049856 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.050078 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.050079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.050079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.050351 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:49.050500 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.050624 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.050745 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260811 10:14:49.059792 32578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:14:49.060308 32578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:49.067767 32578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Generated new TSK 0
I20260811 10:14:49.067952 32578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:14:49.098830 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:49.183159 32588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:49.183382 32588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:49.183421 32588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:49.184578 32588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:49.184640 32588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:49.184681 32588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:14:49.186136 32588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:49.186400 32588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:49.186623 32588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:49.191860 32593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.191954 32596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.191875 32598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:49.192231 32588 server_base.cc:1034] running on GCE node
I20260811 10:14:49.192444 32588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:49.192641 32588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:49.193773 32588 hybrid_clock.cc:648] HybridClock initialized: now 1786443289193745 us; error 43 us; skew 500 ppm
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:49.196377 32588 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:14:49.196780 32588 webserver.cc:460] Webserver started at http://127.8.238.1:41427/ using document root <none> and password file <none>
I20260811 10:14:49.197007 32588 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:49.197072 32588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:49.197214 32588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:49.198215 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "a6c882d261c8465592b28bc255cecd51"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "999b47b215004211e86e50094cedf3ed"
server_key_iv: "61e443e1c50d95d452a5ef525fed4cf1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.198562 32588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "a6c882d261c8465592b28bc255cecd51"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "999b47b215004211e86e50094cedf3ed"
server_key_iv: "61e443e1c50d95d452a5ef525fed4cf1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.199942 32588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:14:49.200637 32608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.200805 32588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:49.200913 32588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "a6c882d261c8465592b28bc255cecd51"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "999b47b215004211e86e50094cedf3ed"
server_key_iv: "61e443e1c50d95d452a5ef525fed4cf1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.200985 32588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.227893 32588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.229903 32588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.230063 32588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.230298 32588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:49.230607 32588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:49.230659 32588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.230697 32588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:49.230724 32588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.237015 32588 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:40427
I20260811 10:14:49.237063 323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:40427 every 8 connection(s)
I20260811 10:14:49.237416 32588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:49.244668 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 32588
I20260811 10:14:49.244807 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:14:49.245052 9144 external_mini_cluster.cc:1428] Setting key b3b16d983f2a683bc2447a2366c7d9c7
I20260811 10:14:49.247049 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.238112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:56897)
0811 10:14:49.238269 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.238319 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.238791 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.238994 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.238997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.239563 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:49.239772 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.239777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.240505 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.240506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.241336 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.241339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.241730 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.241731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.241838 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.242261 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:49.242269 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:49.243784 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:49.245637 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.245644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.245646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.245921 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.246158 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.246159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.246160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.246272 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.246565 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:49.246570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:49.246679 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.246771 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.246888 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260811 10:14:49.247051 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.238150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:56897 (local address 127.8.238.62:39947)
0811 10:14:49.238194 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.238196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.238398 (+ 202us) server_negotiation.cc:407] Connection header received
0811 10:14:49.238879 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.238880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.238890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.238944 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:49.239876 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.240402 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.241433 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.241650 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.241710 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.243887 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:49.243892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:49.243892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:49.243899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:49.245565 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.245990 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.245992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.245993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.246065 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.246345 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.246346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.246346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.246498 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:49.246571 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.246809 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.246917 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:14:49.247627 325 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:49.247782 325 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:49.248001 325 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:49.248550 32489 ts_manager.cc:194] Registered new tserver with Master: a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427)
I20260811 10:14:49.249243 32489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:56897
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:14:49.272478 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:49.355602 330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:49.355849 330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:49.355890 330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:49.357014 330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:49.357079 330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:49.357122 330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:14:49.358579 330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:49.358837 330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:49.359062 330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:49.364032 337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.364158 343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.364058 338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:49.364233 330 server_base.cc:1034] running on GCE node
I20260811 10:14:49.364425 330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:49.364614 330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:49.365767 330 hybrid_clock.cc:648] HybridClock initialized: now 1786443289365755 us; error 33 us; skew 500 ppm
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:49.368438 330 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:14:49.368893 330 webserver.cc:460] Webserver started at http://127.8.238.2:40633/ using document root <none> and password file <none>
I20260811 10:14:49.369131 330 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:49.369176 330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:49.369318 330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:49.370309 330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "d163f91f51a444ad9944cb1f54e26b33"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "11db1f0d632f9adce65e14a798bd7044"
server_key_iv: "cff94d9055d4aed4a4f536cd57527902"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.370654 330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "d163f91f51a444ad9944cb1f54e26b33"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "11db1f0d632f9adce65e14a798bd7044"
server_key_iv: "cff94d9055d4aed4a4f536cd57527902"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.372048 330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:14:49.372825 356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.373034 330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:49.373124 330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "d163f91f51a444ad9944cb1f54e26b33"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "11db1f0d632f9adce65e14a798bd7044"
server_key_iv: "cff94d9055d4aed4a4f536cd57527902"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.373180 330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.394580 330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.396895 330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.397012 330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.397246 330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:49.397595 330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:49.397648 330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.397683 330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:49.397723 330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.403949 330 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:40733
I20260811 10:14:49.404048 535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:40733 every 8 connection(s)
I20260811 10:14:49.404345 330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:14:49.407583 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 330
I20260811 10:14:49.407701 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:14:49.407919 9144 external_mini_cluster.cc:1428] Setting key 3bf135274905b0f6cc743e8db2975a6e
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:49.414518 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.405035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:50717)
0811 10:14:49.405211 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.405277 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.405867 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.406112 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.406115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.406670 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:49.406898 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.406904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.407657 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.407659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.408736 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.408739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.409202 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.409203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.409306 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.409804 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:49.409812 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:49.411141 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:49.413049 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.413056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.413057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.413280 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.413513 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.413514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.413514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.413612 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.413968 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:49.413975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:49.414120 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.414219 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.414360 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260811 10:14:49.414511 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.405079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:50717 (local address 127.8.238.62:39947)
0811 10:14:49.405158 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.405161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.405376 (+ 215us) server_negotiation.cc:407] Connection header received
0811 10:14:49.405963 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.405964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.405974 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.406028 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:49.406988 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.407556 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.408816 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.409117 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.409191 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.411237 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:49.411238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:49.411239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:49.411247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:49.412969 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.413363 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.413364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.413365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.413441 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.413700 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.413701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.413701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.413877 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:49.413957 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.414276 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.414366 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:14:49.415180 537 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:49.415297 537 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:49.415508 537 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:49.415946 32489 ts_manager.cc:194] Registered new tserver with Master: d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733)
I20260811 10:14:49.416334 32489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:50717
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:14:49.436606 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:49.519556 545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:49.519815 545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:49.519860 545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:49.521013 545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:49.521078 545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:49.521124 545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:14:49.522578 545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:49.522842 545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:49.523077 545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:49.528198 552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.528194 551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.528200 557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:49.528566 545 server_base.cc:1034] running on GCE node
I20260811 10:14:49.528796 545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:49.529024 545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:49.530174 545 hybrid_clock.cc:648] HybridClock initialized: now 1786443289530170 us; error 38 us; skew 500 ppm
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:49.533172 545 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:14:49.533646 545 webserver.cc:460] Webserver started at http://127.8.238.3:37113/ using document root <none> and password file <none>
I20260811 10:14:49.533864 545 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:49.533908 545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:49.534055 545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:14:49.535084 545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "b1c1371e581d0c73cd187019cc452233"
server_key_iv: "e6845de10e85c1135dccd0c12b2115b1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.535447 545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "b1c1371e581d0c73cd187019cc452233"
server_key_iv: "e6845de10e85c1135dccd0c12b2115b1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.536827 545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:14:49.537521 566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.537688 545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:49.537793 545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "b1c1371e581d0c73cd187019cc452233"
server_key_iv: "e6845de10e85c1135dccd0c12b2115b1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.537865 545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.546923 545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.548697 545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.548856 545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.549067 545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:49.549365 545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:49.549418 545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.549467 545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:49.549495 545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.555816 545 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33823
I20260811 10:14:49.555826 741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33823 every 8 connection(s)
I20260811 10:14:49.556160 545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:14:49.561575 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 545
I20260811 10:14:49.561650 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:14:49.561801 9144 external_mini_cluster.cc:1428] Setting key 9beb1d3472372659e7325a33e66f0819
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:49.564878 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.556790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:52589 (local address 127.8.238.62:39947)
0811 10:14:49.556833 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.556835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.557064 (+ 229us) server_negotiation.cc:407] Connection header received
0811 10:14:49.557514 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.557515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.557524 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.557581 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:49.558475 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.558968 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.559991 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.560199 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.560255 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.562477 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:49.562479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:49.562480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:49.562488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:49.563727 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.564132 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.564133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.564133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.564190 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.564401 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.564402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.564402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.564507 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:49.564558 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.564755 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.564813 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:14:49.565135 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.556762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:52589)
0811 10:14:49.556938 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.556978 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.557434 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.557653 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.557656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.558215 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:49.558401 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.558407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.559042 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.559043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.559918 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.559920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.560271 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.560272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.560377 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.560873 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:49.560881 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:49.562408 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:49.563841 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.563848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.563850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.564069 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.564245 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:49.564246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:49.564247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:49.564350 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:49.564558 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:49.564563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:49.564668 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.564744 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.564861 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260811 10:14:49.565630 745 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:49.565733 745 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:49.565910 745 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:49.566284 32489 ts_manager.cc:194] Registered new tserver with Master: 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
I20260811 10:14:49.566636 32489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:52589
I20260811 10:14:49.573854 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:14:49.581194 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.575206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.8.238.62:39947)
0811 10:14:49.575262 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.575263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.575381 (+ 118us) server_negotiation.cc:407] Connection header received
0811 10:14:49.575420 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.575421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.575429 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.575485 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:49.576128 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.576611 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.577330 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.577519 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.577576 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.578676 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:49.578677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:49.578677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:49.578683 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:49.580200 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.580531 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.580532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.580533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.580595 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:49.580754 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:49.580755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:49.580756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:49.580894 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:49.580971 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.581066 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.581130 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:14:49.582831 32489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:14:49.584616 32489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:14:49.592067 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:40427 (local address 127.0.0.1:34032)
0811 10:14:49.587428 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.587438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.587480 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.587779 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.587782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.587785 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:49.587856 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.587857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589274 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.589275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.590778 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.590779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.591756 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.591757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.591850 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.591865 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.591917 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.591985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":116,"thread_start_us":90,"threads_started":1}
I20260811 10:14:49.592212 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.8.238.3:33823)
0811 10:14:49.587726 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.587730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.587767 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:14:49.587806 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.587808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.587823 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.587899 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:49.588017 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589134 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.590880 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.591623 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.591975 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.592045 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.592072 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.592123 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
I20260811 10:14:49.592345 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33823 (local address 127.0.0.1:38472)
0811 10:14:49.587430 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.587442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.587498 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.587900 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.587901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.587903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:49.587958 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.587961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589273 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.589274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.590776 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.590779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.591731 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.591761 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.591850 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.591866 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.591927 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.592273 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":406,"thread_start_us":86,"threads_started":1}
I20260811 10:14:49.592895 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.8.238.1:40427)
0811 10:14:49.587560 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.587565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.587574 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:14:49.587618 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.587620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.587654 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.587767 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:49.588053 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589143 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.590880 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.591623 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.592681 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.592744 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.592770 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.592813 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":93,"threads_started":1}
I20260811 10:14:49.593052 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.8.238.2:40733)
0811 10:14:49.587640 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.587645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.587655 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:14:49.587699 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.587701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.587717 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.587986 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:49.588128 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589151 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.591863 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.592625 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.592721 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.592781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.592871 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.592949 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":86,"threads_started":1}
I20260811 10:14:49.593221 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.587091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:40733 (local address 127.0.0.1:50696)
0811 10:14:49.587273 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.587319 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.587387 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.587841 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.587842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.587844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:49.587906 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.587908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.589283 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.589284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.591683 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.591686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.592724 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.592725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.592786 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.592791 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.592849 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.592890 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":764,"thread_start_us":61,"threads_started":1}
I20260811 10:14:49.593101 641 tablet_service.cc:1467] Processing CreateTablet for tablet 377512e1973742869e3e605edef73ae7 (DEFAULT_TABLE table=test-table [id=c1ae543d0cb246ae8800670bc6042e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:49.593489 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377512e1973742869e3e605edef73ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:49.593498 32690 tablet_service.cc:1467] Processing CreateTablet for tablet 377512e1973742869e3e605edef73ae7 (DEFAULT_TABLE table=test-table [id=c1ae543d0cb246ae8800670bc6042e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:49.593825 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377512e1973742869e3e605edef73ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:49.597141 771 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Bootstrap starting.
I20260811 10:14:49.597915 771 tablet_bootstrap.cc:654] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:49.598017 429 tablet_service.cc:1467] Processing CreateTablet for tablet 377512e1973742869e3e605edef73ae7 (DEFAULT_TABLE table=test-table [id=c1ae543d0cb246ae8800670bc6042e39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:49.598428 429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377512e1973742869e3e605edef73ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:49.598567 771 log.cc:824] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:49.598918 770 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Bootstrap starting.
I20260811 10:14:49.599308 771 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: No bootstrap required, opened a new log
I20260811 10:14:49.599396 771 ts_tablet_manager.cc:1397] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:49.599563 770 tablet_bootstrap.cc:654] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:49.600231 770 log.cc:824] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:49.601042 770 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: No bootstrap required, opened a new log
I20260811 10:14:49.601115 770 ts_tablet_manager.cc:1397] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:49.601343 771 raft_consensus.cc:348] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.601511 771 raft_consensus.cc:374] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:49.601557 771 raft_consensus.cc:729] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf77aa1b428469c8645d3a1e7679a1c, State: Initialized, Role: FOLLOWER
I20260811 10:14:49.601670 771 consensus_queue.cc:260] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.601990 771 ts_tablet_manager.cc:1428] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:14:49.601998 745 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
I20260811 10:14:49.602298 775 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Bootstrap starting.
I20260811 10:14:49.602946 775 tablet_bootstrap.cc:654] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:49.603291 770 raft_consensus.cc:348] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.603500 770 raft_consensus.cc:374] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:49.603542 770 raft_consensus.cc:729] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c882d261c8465592b28bc255cecd51, State: Initialized, Role: FOLLOWER
I20260811 10:14:49.603612 775 log.cc:824] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:49.603663 770 consensus_queue.cc:260] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.604082 770 ts_tablet_manager.cc:1428] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 10:14:49.604220 325 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
I20260811 10:14:49.604486 775 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: No bootstrap required, opened a new log
I20260811 10:14:49.604557 775 ts_tablet_manager.cc:1397] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:49.606191 775 raft_consensus.cc:348] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.606316 775 raft_consensus.cc:374] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:49.606338 775 raft_consensus.cc:729] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d163f91f51a444ad9944cb1f54e26b33, State: Initialized, Role: FOLLOWER
I20260811 10:14:49.606422 775 consensus_queue.cc:260] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.606611 775 ts_tablet_manager.cc:1428] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:49.606663 537 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
I20260811 10:14:49.621068 777 raft_consensus.cc:482] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:49.621171 777 raft_consensus.cc:504] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.621464 777 leader_election.cc:288] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
I20260811 10:14:49.624316 764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.621720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:39491 (local address 127.8.238.2:40733)
0811 10:14:49.621772 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.621774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.621784 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:14:49.621813 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.621814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.621826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.621922 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:49.622101 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.622577 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.623517 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.623971 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.624052 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.624064 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.624210 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.624248 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:14:49.624317 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.621644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:40733 (local address 127.8.238.1:39491)
0811 10:14:49.621696 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.621708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.621754 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.621944 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.621946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.621948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:49.622022 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.622025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.622661 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.622662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.623420 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.623422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.624051 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.624052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.624137 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.624140 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.624203 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.624243 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 10:14:49.624490 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.621880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:50517 (local address 127.8.238.3:33823)
0811 10:14:49.621949 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.621951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.622028 (+ 77us) server_negotiation.cc:407] Connection header received
0811 10:14:49.622095 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.622096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.622105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.622156 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:49.622311 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.622768 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.623690 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.624138 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.624221 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.624231 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.624387 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.624426 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:14:49.624503 781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.621776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33823 (local address 127.8.238.1:50517)
0811 10:14:49.621958 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.621967 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:49.622042 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:49.622169 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:49.622170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:49.622172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:49.622246 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.622248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.622854 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.622856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.623601 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:49.623603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.624225 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.624227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:49.624300 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.624304 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:14:49.624377 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 10:14:49.624424 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260811 10:14:49.624704 456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377512e1973742869e3e605edef73ae7" candidate_uuid: "a6c882d261c8465592b28bc255cecd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d163f91f51a444ad9944cb1f54e26b33" is_pre_election: true
I20260811 10:14:49.624873 456 raft_consensus.cc:2454] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c882d261c8465592b28bc255cecd51 in term 0.
I20260811 10:14:49.625142 32611 leader_election.cc:302] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c882d261c8465592b28bc255cecd51, d163f91f51a444ad9944cb1f54e26b33; no voters:
I20260811 10:14:49.625274 777 raft_consensus.cc:2781] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:14:49.625229 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377512e1973742869e3e605edef73ae7" candidate_uuid: "a6c882d261c8465592b28bc255cecd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" is_pre_election: true
I20260811 10:14:49.625331 777 raft_consensus.cc:482] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:14:49.625349 777 raft_consensus.cc:3037] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:49.625392 670 raft_consensus.cc:2454] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c882d261c8465592b28bc255cecd51 in term 0.
I20260811 10:14:49.626142 777 raft_consensus.cc:504] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.626276 777 leader_election.cc:288] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 1 election: Requested vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
I20260811 10:14:49.626451 456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377512e1973742869e3e605edef73ae7" candidate_uuid: "a6c882d261c8465592b28bc255cecd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d163f91f51a444ad9944cb1f54e26b33"
I20260811 10:14:49.626477 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377512e1973742869e3e605edef73ae7" candidate_uuid: "a6c882d261c8465592b28bc255cecd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
I20260811 10:14:49.626524 456 raft_consensus.cc:3037] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:49.626582 670 raft_consensus.cc:3037] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:49.627274 456 raft_consensus.cc:2454] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c882d261c8465592b28bc255cecd51 in term 1.
I20260811 10:14:49.627336 670 raft_consensus.cc:2454] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c882d261c8465592b28bc255cecd51 in term 1.
I20260811 10:14:49.627471 32611 leader_election.cc:302] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c882d261c8465592b28bc255cecd51, d163f91f51a444ad9944cb1f54e26b33; no voters:
I20260811 10:14:49.627602 777 raft_consensus.cc:2781] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:49.627795 777 raft_consensus.cc:686] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 LEADER]: Becoming Leader. State: Replica: a6c882d261c8465592b28bc255cecd51, State: Running, Role: LEADER
I20260811 10:14:49.627911 777 consensus_queue.cc:237] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:49.628723 32489 catalog_manager.cc:5495] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6c882d261c8465592b28bc255cecd51 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "a6c882d261c8465592b28bc255cecd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } health_report { overall_health: UNKNOWN } } }
I20260811 10:14:49.641400 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.639106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.8.238.1:40427)
0811 10:14:49.639158 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:49.639160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:49.639170 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:14:49.639226 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:49.639226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:49.639244 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:49.639301 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:14:49.639395 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.639895 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.640681 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:49.640864 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:49.640910 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:49.641043 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:14:49.641136 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:14:49.641195 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:49.641224 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 10:14:49.641238 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:14:49.642992 670 raft_consensus.cc:1261] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Refusing update from remote peer a6c882d261c8465592b28bc255cecd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:49.643038 456 raft_consensus.cc:1261] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Refusing update from remote peer a6c882d261c8465592b28bc255cecd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:49.643322 777 consensus_queue.cc:1035] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:49.643481 782 consensus_queue.cc:1035] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:49.645818 788 mvcc.cc:205] Tried to move back new op lower bound from 7317271714375225344 to 7317271714316427264. Current Snapshot: MvccSnapshot[applied={T|T < 7317271714375225344}]
I20260811 10:14:49.646621 789 mvcc.cc:205] Tried to move back new op lower bound from 7317271714375225344 to 7317271714316427264. Current Snapshot: MvccSnapshot[applied={T|T < 7317271714375225344 or (T in {7317271714375225344})}]
I20260811 10:14:49.646742 790 mvcc.cc:205] Tried to move back new op lower bound from 7317271714375225344 to 7317271714316427264. Current Snapshot: MvccSnapshot[applied={T|T < 7317271714375225344}]
I20260811 10:14:49.648576 32489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:49.648680 32489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:49.649452 32489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Sending DeleteTablet for 3 replicas of tablet 377512e1973742869e3e605edef73ae7
I20260811 10:14:49.649801 429 tablet_service.cc:1514] Processing DeleteTablet for tablet 377512e1973742869e3e605edef73ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:49 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:50696
I20260811 10:14:49.649839 641 tablet_service.cc:1514] Processing DeleteTablet for tablet 377512e1973742869e3e605edef73ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:49 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:38472
I20260811 10:14:49.649904 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 32588
I20260811 10:14:49.650017 32690 tablet_service.cc:1514] Processing DeleteTablet for tablet 377512e1973742869e3e605edef73ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:49 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34032
I20260811 10:14:49.655329 801 tablet_replica.cc:330] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: stopping tablet replica
I20260811 10:14:49.656247 801 raft_consensus.cc:2229] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:49.656376 801 raft_consensus.cc:2258] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:49.657997 801 ts_tablet_manager.cc:1905] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:14:49.659381 802 tablet_replica.cc:330] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: stopping tablet replica
I20260811 10:14:49.659528 802 raft_consensus.cc:2229] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:49.659662 802 raft_consensus.cc:2258] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:49.660244 801 ts_tablet_manager.cc:1918] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:49.660333 801 log.cc:1196] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/377512e1973742869e3e605edef73ae7
I20260811 10:14:49.660629 801 ts_tablet_manager.cc:1939] T 377512e1973742869e3e605edef73ae7 P d163f91f51a444ad9944cb1f54e26b33: Deleting consensus metadata
I20260811 10:14:49.661118 32468 catalog_manager.cc:4841] TS d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): tablet 377512e1973742869e3e605edef73ae7 (table test-table [id=c1ae543d0cb246ae8800670bc6042e39]) successfully deleted
I20260811 10:14:49.664189 802 ts_tablet_manager.cc:1905] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 10:14:49.665155 32466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:40427 (error 108)
I20260811 10:14:49.665365 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 330
I20260811 10:14:49.665966 802 ts_tablet_manager.cc:1918] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:49.666042 802 log.cc:1196] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/377512e1973742869e3e605edef73ae7
I20260811 10:14:49.666292 802 ts_tablet_manager.cc:1939] T 377512e1973742869e3e605edef73ae7 P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting consensus metadata
I20260811 10:14:49.675655 763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:49.675504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:40427 (local address 127.0.0.1:34042)
0811 10:14:49.675561 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:49.675602 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:14:49.675731 32468 catalog_manager.cc:4841] TS 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): tablet 377512e1973742869e3e605edef73ae7 (table test-table [id=c1ae543d0cb246ae8800670bc6042e39]) successfully deleted
W20260811 10:14:49.675819 32466 catalog_manager.cc:4584] TS a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 377512e1973742869e3e605edef73ae7: Network error: Client connection negotiation failed: client connection to 127.8.238.1:40427: connect: Connection refused (error 111)
I20260811 10:14:49.680179 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 545
I20260811 10:14:49.687999 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 32438
I20260811 10:14:49.699818 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--webserver_interface=127.8.238.62
--webserver_port=35565
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:49.783002 806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:49.783217 806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:49.783241 806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:49.784441 806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:49.784489 806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:49.784504 806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:49.784518 806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:49.784530 806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:49.786022 806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=35565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:49.786281 806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:49.786536 806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:14:49.796461 806 server_base.cc:1034] running on GCE node
W20260811 10:14:49.796451 821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.796455 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.796617 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:49.796983 806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:49.797317 806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:49.798480 806 hybrid_clock.cc:648] HybridClock initialized: now 1786443289798476 us; error 34 us; skew 500 ppm
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:49.801118 806 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:49.801517 806 webserver.cc:460] Webserver started at http://127.8.238.62:35565/ using document root <none> and password file <none>
I20260811 10:14:49.801777 806 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:49.801882 806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:49.803319 806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:14:49.804062 830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.804273 806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:49.804390 806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e1fcc1d4d03e4264a72515238fb05265"
format_stamp: "Formatted at 2026-08-11 10:14:48 on dist-test-slave-7f01"
server_key: "eeb89f362d4ad7ac2455c6e94a9c7627"
server_key_iv: "74be3b618f57feb84465f15d49dc9305"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.804855 806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.828963 806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.830981 806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.831143 806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.835598 806 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39947
I20260811 10:14:49.835644 918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39947 every 8 connection(s)
I20260811 10:14:49.836098 806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:49.837692 919 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:14:49.838335 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap starting.
I20260811 10:14:49.845229 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 806
I20260811 10:14:49.845633 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:40427
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=41427
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:49.848529 919 log.cc:824] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:49.850476 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:14:49.850772 919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap complete.
I20260811 10:14:49.853229 919 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.853487 919 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Initialized, Role: FOLLOWER
I20260811 10:14:49.853691 919 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.853787 919 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:49.853849 919 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:49.853899 919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:14:49.854926 919 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.855048 919 leader_election.cc:302] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1fcc1d4d03e4264a72515238fb05265; no voters:
I20260811 10:14:49.855242 919 leader_election.cc:288] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:14:49.855293 924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:14:49.855422 924 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 LEADER]: Becoming Leader. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Running, Role: LEADER
I20260811 10:14:49.855526 919 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:49.855509 924 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:49.855813 924 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1fcc1d4d03e4264a72515238fb05265. Latest consensus state: current_term: 2 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:49.855896 924 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:49.855816 925 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:49.855952 925 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:49.856102 930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:49.856978 930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:49.857116 930 catalog_manager.cc:1259] Loaded cluster ID: 8a60ef7cae434cfb8dc9a9380d9f1fa7
I20260811 10:14:49.857409 930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:14:49.857654 930 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 10:14:49.860438 930 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 10:14:49.868811 930 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:14:49.869036 930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Loaded TSK: 0
I20260811 10:14:49.869258 930 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:14:49.965248 922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:49.965543 922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:49.965590 922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:49.967303 922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:49.967371 922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:49.967415 922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:14:49.969604 922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:40427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=41427
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:49.969867 922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:49.970172 922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:49.975668 976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.975729 972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:49.975667 971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:49.976432 922 server_base.cc:1034] running on GCE node
I20260811 10:14:49.976670 922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:49.976958 922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:49.978138 922 hybrid_clock.cc:648] HybridClock initialized: now 1786443289978115 us; error 39 us; skew 500 ppm
Aug 11 10:14:49 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:49.981199 922 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:14:49.981698 922 webserver.cc:460] Webserver started at http://127.8.238.1:41427/ using document root <none> and password file <none>
I20260811 10:14:49.981948 922 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:49.982033 922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:49.983469 922 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:49.984292 984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:49.984472 922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:14:49.984577 922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "a6c882d261c8465592b28bc255cecd51"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "999b47b215004211e86e50094cedf3ed"
server_key_iv: "61e443e1c50d95d452a5ef525fed4cf1"
server_key_version: "encryptionkey@0"
I20260811 10:14:49.984983 922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:49.994751 922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:49.997300 922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:49.997457 922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:49.997705 922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:49.998208 992 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:14:49.999728 922 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:14:49.999841 922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:14:49.999882 922 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:14:50.000787 922 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:14:50.000845 922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:14:50.000902 992 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Bootstrap starting.
I20260811 10:14:50.008539 922 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:40427
I20260811 10:14:50.009074 922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:14:50.012100 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 922
I20260811 10:14:50.012410 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:40733
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=40633
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:50.013309 1172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:40427 every 8 connection(s)
I20260811 10:14:50.015981 992 log.cc:824] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Log is configured to *not* fsync() on all Append() calls
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443289, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:50.022146 992 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:14:50.022507 992 tablet_bootstrap.cc:492] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Bootstrap complete.
I20260811 10:14:50.023974 992 ts_tablet_manager.cc:1397] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.015s
I20260811 10:14:50.025255 992 raft_consensus.cc:348] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:50.025563 992 raft_consensus.cc:729] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c882d261c8465592b28bc255cecd51, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.025717 992 consensus_queue.cc:260] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } }
I20260811 10:14:50.025995 992 ts_tablet_manager.cc:1428] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:14:50.027992 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.009814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:59573)
0811 10:14:50.011902 (+ 2088us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.011960 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.012600 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.013204 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.013209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.013688 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:50.013975 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.013984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.015713 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.015715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.017004 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.017009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.017899 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.017900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.018064 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.018735 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:50.018746 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:50.022879 (+ 4133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:50.026160 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.026166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.026168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.026449 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.026757 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.026758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.026759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.026891 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.027346 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:50.027352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:50.027500 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.027598 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.027789 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2037,"thread_start_us":136,"threads_started":1}
I20260811 10:14:50.028100 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.009970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:59573 (local address 127.8.238.62:39947)
0811 10:14:50.011905 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.011918 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.012095 (+ 177us) server_negotiation.cc:407] Connection header received
0811 10:14:50.012913 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.012934 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.013001 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.013132 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:50.014124 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.015567 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.017105 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.017757 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.017905 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.023029 (+ 5124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:50.023034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:50.023034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:50.023048 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:50.025967 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.026552 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.026555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.026555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.026678 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.026990 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.026991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.026991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.027275 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:50.027357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.027684 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.027891 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1867,"thread_start_us":98,"threads_started":1}
I20260811 10:14:50.028940 1173 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:50.029042 1173 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:50.029214 1173 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:50.030020 854 ts_manager.cc:194] Registered new tserver with Master: a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427)
I20260811 10:14:50.031188 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:59573
I20260811 10:14:50.032220 1173 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
I20260811 10:14:50.036242 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.030614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:40427 (local address 127.0.0.1:34050)
0811 10:14:50.031860 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.031896 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.031956 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.032255 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.032257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.032259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:50.032343 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.032345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.033656 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.033658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.034893 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.034898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.035924 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.035925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.036026 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.036039 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.036110 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.036162 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":111,"threads_started":1}
I20260811 10:14:50.036509 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.030889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.8.238.1:40427)
0811 10:14:50.032100 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.032105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.032113 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:14:50.032146 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.032148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.032163 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.032412 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:50.032456 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.033562 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.035004 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.035727 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.035820 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.035887 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.036153 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.036211 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":70,"threads_started":1}
I20260811 10:14:50.036965 1067 tablet_service.cc:1514] Processing DeleteTablet for tablet 377512e1973742869e3e605edef73ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:49 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34050
I20260811 10:14:50.037524 1186 tablet_replica.cc:330] stopping tablet replica
I20260811 10:14:50.037640 1186 raft_consensus.cc:2229] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:50.037706 1186 raft_consensus.cc:2258] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:50.038014 1186 ts_tablet_manager.cc:1905] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:14:50.040392 1186 ts_tablet_manager.cc:1918] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:50.040473 1186 log.cc:1196] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/377512e1973742869e3e605edef73ae7
I20260811 10:14:50.040709 1186 ts_tablet_manager.cc:1939] T 377512e1973742869e3e605edef73ae7 P a6c882d261c8465592b28bc255cecd51: Deleting consensus metadata
I20260811 10:14:50.041204 832 catalog_manager.cc:4841] TS a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427): tablet 377512e1973742869e3e605edef73ae7 (table test-table [id=c1ae543d0cb246ae8800670bc6042e39]) successfully deleted
W20260811 10:14:50.104524 1177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:50.104730 1177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:50.104753 1177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:50.105836 1177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:50.105870 1177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:50.105894 1177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:14:50.107266 1177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:40733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=40633
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:50.107493 1177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:50.107703 1177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:50.112792 1200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.112840 1194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.112845 1193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.112879 1177 server_base.cc:1034] running on GCE node
I20260811 10:14:50.113205 1177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:50.113462 1177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:50.114626 1177 hybrid_clock.cc:648] HybridClock initialized: now 1786443290114625 us; error 41 us; skew 500 ppm
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:50.117223 1177 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:14:50.117646 1177 webserver.cc:460] Webserver started at http://127.8.238.2:40633/ using document root <none> and password file <none>
I20260811 10:14:50.117869 1177 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:50.117951 1177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:50.119320 1177 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:50.120009 1211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.120183 1177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:50.120306 1177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "d163f91f51a444ad9944cb1f54e26b33"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "11db1f0d632f9adce65e14a798bd7044"
server_key_iv: "cff94d9055d4aed4a4f536cd57527902"
server_key_version: "encryptionkey@0"
I20260811 10:14:50.120669 1177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:50.133817 1177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:50.135620 1177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:50.135838 1177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:50.136070 1177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:50.136387 1177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:50.136443 1177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.136483 1177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:50.136510 1177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.142686 1177 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:40733
I20260811 10:14:50.142791 1399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:40733 every 8 connection(s)
I20260811 10:14:50.143103 1177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:50.150343 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 1177
I20260811 10:14:50.150591 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:33823
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=37113
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:50.152633 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.143883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:54063 (local address 127.8.238.62:39947)
0811 10:14:50.143932 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.143934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.144165 (+ 231us) server_negotiation.cc:407] Connection header received
0811 10:14:50.144727 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.144729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.144743 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.144797 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:50.145753 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.146272 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.147288 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.147510 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.147587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.149753 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:50.149755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:50.149756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:50.149763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:50.151230 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.151629 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.151630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.151631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.151696 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.151977 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.151978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.151979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.152098 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:50.152172 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.152447 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.152539 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:14:50.152818 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.143903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:54063)
0811 10:14:50.144048 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.144085 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.144610 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.144867 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.144871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.145455 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:50.145647 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.145652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.146378 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.146380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.147189 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.147191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.147595 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.147597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.147721 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.148285 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:50.148294 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:50.149656 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:50.151316 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.151324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.151326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.151545 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.151785 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.151786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.151787 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.151903 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.152163 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:50.152171 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:50.152317 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.152424 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.152579 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260811 10:14:50.153517 1400 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:50.153599 1400 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:50.153811 1400 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:50.154244 854 ts_manager.cc:194] Registered new tserver with Master: d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733)
I20260811 10:14:50.154784 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:54063
W20260811 10:14:50.236027 1407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:50.236228 1407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:50.236258 1407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:50.237434 1407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:50.237476 1407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:50.237502 1407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:14:50.238873 1407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:33823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=37113
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:50.239063 1407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:50.239254 1407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:50.244217 1426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.244233 1423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.244278 1420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.244263 1407 server_base.cc:1034] running on GCE node
I20260811 10:14:50.244638 1407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:50.244911 1407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:50.246057 1407 hybrid_clock.cc:648] HybridClock initialized: now 1786443290246035 us; error 35 us; skew 500 ppm
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:50.248656 1407 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:14:50.249051 1407 webserver.cc:460] Webserver started at http://127.8.238.3:37113/ using document root <none> and password file <none>
I20260811 10:14:50.249248 1407 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:50.249301 1407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:50.250578 1407 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:50.251186 1439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.251374 1407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:50.251466 1407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "b1c1371e581d0c73cd187019cc452233"
server_key_iv: "e6845de10e85c1135dccd0c12b2115b1"
server_key_version: "encryptionkey@0"
I20260811 10:14:50.251844 1407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:50.260449 1407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:50.262393 1407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:50.262502 1407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:50.262671 1407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:50.262923 1407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:50.262952 1407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.262974 1407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:50.262986 1407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.269100 1407 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33823
I20260811 10:14:50.269168 1628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33823 every 8 connection(s)
I20260811 10:14:50.269511 1407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:14:50.277256 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 1407
I20260811 10:14:50.278748 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.270198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:42541 (local address 127.8.238.62:39947)
0811 10:14:50.270247 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.270249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.270447 (+ 198us) server_negotiation.cc:407] Connection header received
0811 10:14:50.270923 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.270924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.270934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.270991 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:50.271943 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.272457 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.273435 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.273635 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.273692 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.275912 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:50.275914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:50.275914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:50.275921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:50.277211 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.277695 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.277696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.277696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.277775 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.278107 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.278107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.278108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.278221 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:50.278270 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.278615 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.278683 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:14:50.279006 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.270161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:42541)
0811 10:14:50.270322 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.270362 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.270836 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.271039 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.271042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.271600 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:50.271845 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.271850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.272540 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.272541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.273351 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.273354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.273708 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.273709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.273895 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.274384 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:50.274393 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:50.275815 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:50.277326 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.277334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.277336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.277594 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.277862 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:50.277863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:50.277863 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:50.278004 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:50.278280 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:50.278287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:50.278461 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.278585 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.278756 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260811 10:14:50.279500 1629 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39947
I20260811 10:14:50.279600 1629 heartbeater.cc:464] Registering TS with master...
I20260811 10:14:50.279804 1629 heartbeater.cc:511] Master 127.8.238.62:39947 requested a full tablet report, sending...
I20260811 10:14:50.280264 854 ts_manager.cc:194] Registered new tserver with Master: 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
I20260811 10:14:50.280791 854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:42541
I20260811 10:14:50.284458 1633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.277682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.8.238.62:39947)
0811 10:14:50.277805 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.277806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.277903 (+ 97us) server_negotiation.cc:407] Connection header received
0811 10:14:50.277978 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.277978 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.277988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.278043 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:50.278936 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.279763 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.280793 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.281099 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.281170 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.282275 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:50.282277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:50.282277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:50.282283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:50.283475 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.283856 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.283856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.283857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.283925 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:50.284093 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:50.284094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:50.284094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:50.284190 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:50.284309 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.284335 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.284397 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260811 10:14:50.285168 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:14:50.287873 1633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.285500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.8.238.62:39947)
0811 10:14:50.285537 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.285538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.285650 (+ 112us) server_negotiation.cc:407] Connection header received
0811 10:14:50.285699 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.285699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.285729 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.285783 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:14:50.285862 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.286292 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.287052 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.287236 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.287352 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.287534 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:14:50.287632 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:14:50.287712 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.287780 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.287801 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:14:50.288141 854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:14:50.289752 854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:14:50.291968 1067 tablet_service.cc:1467] Processing CreateTablet for tablet 04809324126645e79cd204957b7bea7b (DEFAULT_TABLE table=test-table [id=dbb3def92a8443ffb713c8bcae469489]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:50.292184 1067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04809324126645e79cd204957b7bea7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:50.293583 992 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Bootstrap starting.
I20260811 10:14:50.293998 992 tablet_bootstrap.cc:654] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:50.294713 992 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: No bootstrap required, opened a new log
I20260811 10:14:50.294771 992 ts_tablet_manager.cc:1397] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:14:50.294889 992 raft_consensus.cc:348] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.294965 992 raft_consensus.cc:374] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:50.295017 992 raft_consensus.cc:729] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c882d261c8465592b28bc255cecd51, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.295059 992 consensus_queue.cc:260] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.295141 992 ts_tablet_manager.cc:1428] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.297462 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.293634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38480 (local address 127.8.238.3:33823)
0811 10:14:50.293818 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.293824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.293835 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:14:50.293877 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.293879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.293903 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.293998 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:50.294154 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.295229 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.296614 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.297104 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.297164 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.297239 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.297337 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.297378 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260811 10:14:50.297478 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.291607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33823 (local address 127.0.0.1:38480)
0811 10:14:50.291793 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.291804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.291857 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.294010 (+ 2153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.294011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.294013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:50.294058 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.294061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.295341 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.295343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.296536 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.296538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.297192 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.297193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.297256 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.297260 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.297319 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.297370 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":9,"thread_start_us":102,"threads_started":1}
I20260811 10:14:50.297493 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.291623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.8.238.2:40733)
0811 10:14:50.293859 (+ 2236us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.293862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.293872 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:14:50.293906 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.293907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.293920 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.294148 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:50.294222 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.295313 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.296641 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.297115 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.297206 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.297262 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.297366 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.297410 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2198,"thread_start_us":157,"threads_started":1}
I20260811 10:14:50.297508 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.291508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:40733 (local address 127.0.0.1:50712)
0811 10:14:50.291545 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.291555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.291588 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.294044 (+ 2456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.294045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.294046 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:50.294076 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.294078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.295439 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.295439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.296536 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.296538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.297219 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.297220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.297287 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.297291 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.297342 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.297399 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":25}
I20260811 10:14:50.298156 1289 tablet_service.cc:1467] Processing CreateTablet for tablet 04809324126645e79cd204957b7bea7b (DEFAULT_TABLE table=test-table [id=dbb3def92a8443ffb713c8bcae469489]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:50.298211 1529 tablet_service.cc:1467] Processing CreateTablet for tablet 04809324126645e79cd204957b7bea7b (DEFAULT_TABLE table=test-table [id=dbb3def92a8443ffb713c8bcae469489]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:14:50.298506 1289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04809324126645e79cd204957b7bea7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:50.298527 1529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04809324126645e79cd204957b7bea7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:14:50.301196 1647 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Bootstrap starting.
I20260811 10:14:50.301409 1650 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Bootstrap starting.
I20260811 10:14:50.301903 1647 tablet_bootstrap.cc:654] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:50.302057 1650 tablet_bootstrap.cc:654] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Neither blocks nor log segments found. Creating new log.
I20260811 10:14:50.302637 1647 log.cc:824] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:50.302697 1650 log.cc:824] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:50.303504 1650 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: No bootstrap required, opened a new log
I20260811 10:14:50.303504 1647 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: No bootstrap required, opened a new log
I20260811 10:14:50.303579 1650 ts_tablet_manager.cc:1397] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:50.303579 1647 ts_tablet_manager.cc:1397] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:14:50.305577 1650 raft_consensus.cc:348] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.305577 1647 raft_consensus.cc:348] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.305768 1650 raft_consensus.cc:374] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:50.305768 1647 raft_consensus.cc:374] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:14:50.305809 1650 raft_consensus.cc:729] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d163f91f51a444ad9944cb1f54e26b33, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.305809 1647 raft_consensus.cc:729] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf77aa1b428469c8645d3a1e7679a1c, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.305940 1650 consensus_queue.cc:260] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.305940 1647 consensus_queue.cc:260] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.306233 1650 ts_tablet_manager.cc:1428] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:14:50.306233 1647 ts_tablet_manager.cc:1428] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:14:50.306247 1629 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
I20260811 10:14:50.306330 1400 heartbeater.cc:503] Master 127.8.238.62:39947 was elected leader, sending a full tablet report...
W20260811 10:14:50.393692 1401 tablet.cc:2369] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:14:50.420769 1657 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:50.420980 1657 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.421404 1657 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427)
I20260811 10:14:50.424305 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.421586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:40427 (local address 127.8.238.3:34579)
0811 10:14:50.421659 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.421672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.421716 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.421890 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.421891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.421892 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:50.421976 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.421978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.422632 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.422634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.423435 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.423438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.424033 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.424034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.424119 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.424122 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.424187 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.424228 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 10:14:50.424299 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.421683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:34579 (local address 127.8.238.1:40427)
0811 10:14:50.421733 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.421735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.421744 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:14:50.421808 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.421808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.421819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.421885 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:50.422050 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.422536 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.423522 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.423951 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.424034 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.424045 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.424193 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.424232 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:14:50.424579 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.421686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:49321 (local address 127.8.238.2:40733)
0811 10:14:50.421759 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.421761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.421892 (+ 131us) server_negotiation.cc:407] Connection header received
0811 10:14:50.421967 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.421968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.421981 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.422054 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:14:50.422213 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.422755 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.423698 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.424184 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.424266 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.424276 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.424463 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.424514 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:14:50.424642 1096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04809324126645e79cd204957b7bea7b" candidate_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c882d261c8465592b28bc255cecd51" is_pre_election: true
I20260811 10:14:50.424834 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.421598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:40733 (local address 127.8.238.3:49321)
0811 10:14:50.421820 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.421828 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.421906 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.422056 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:50.422057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:50.422059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:14:50.422137 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.422139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.422851 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.422852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.423604 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:50.423607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.424267 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.424269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:50.424342 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.424364 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 10:14:50.424699 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 10:14:50.424744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":124,"threads_started":1}
I20260811 10:14:50.424840 1096 raft_consensus.cc:2454] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf77aa1b428469c8645d3a1e7679a1c in term 0.
I20260811 10:14:50.425100 1440 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf77aa1b428469c8645d3a1e7679a1c, a6c882d261c8465592b28bc255cecd51; no voters:
I20260811 10:14:50.425207 1320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04809324126645e79cd204957b7bea7b" candidate_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d163f91f51a444ad9944cb1f54e26b33" is_pre_election: true
I20260811 10:14:50.425236 1657 raft_consensus.cc:2781] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:14:50.425298 1657 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:14:50.425359 1657 raft_consensus.cc:3037] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:50.425364 1320 raft_consensus.cc:2454] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cf77aa1b428469c8645d3a1e7679a1c in term 0.
I20260811 10:14:50.426133 1657 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.426285 1657 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [CANDIDATE]: Term 1 election: Requested vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427)
I20260811 10:14:50.426414 1320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04809324126645e79cd204957b7bea7b" candidate_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d163f91f51a444ad9944cb1f54e26b33"
I20260811 10:14:50.426448 1096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04809324126645e79cd204957b7bea7b" candidate_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6c882d261c8465592b28bc255cecd51"
I20260811 10:14:50.426494 1320 raft_consensus.cc:3037] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:50.426539 1096 raft_consensus.cc:3037] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:14:50.427274 1320 raft_consensus.cc:2454] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf77aa1b428469c8645d3a1e7679a1c in term 1.
I20260811 10:14:50.427455 1443 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33; no voters:
I20260811 10:14:50.427819 1657 raft_consensus.cc:2781] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:14:50.427899 1096 raft_consensus.cc:2454] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cf77aa1b428469c8645d3a1e7679a1c in term 1.
I20260811 10:14:50.427979 1657 raft_consensus.cc:686] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 LEADER]: Becoming Leader. State: Replica: 0cf77aa1b428469c8645d3a1e7679a1c, State: Running, Role: LEADER
I20260811 10:14:50.428094 1657 consensus_queue.cc:237] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.429028 854 catalog_manager.cc:5495] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3). New cstate: current_term: 1 leader_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } health_report { overall_health: UNKNOWN } } }
I20260811 10:14:50.444063 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.441904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.8.238.3:33823)
0811 10:14:50.441964 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:50.441965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:50.441973 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:14:50.442002 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:50.442003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:50.442018 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:50.442077 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:14:50.442169 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.442625 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.443324 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:50.443521 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:50.443575 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:50.443702 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:14:50.443849 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:14:50.443921 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:50.443975 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 10:14:50.443993 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:14:50.445930 1320 raft_consensus.cc:1261] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Refusing update from remote peer 0cf77aa1b428469c8645d3a1e7679a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:50.445930 1096 raft_consensus.cc:1261] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Refusing update from remote peer 0cf77aa1b428469c8645d3a1e7679a1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:14:50.446336 1662 consensus_queue.cc:1035] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:50.446518 1657 consensus_queue.cc:1035] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:14:50.448666 1665 mvcc.cc:205] Tried to move back new op lower bound from 7317271717663674368 to 7317271717594255360. Current Snapshot: MvccSnapshot[applied={T|T < 7317271717663674368}]
I20260811 10:14:50.449005 1666 mvcc.cc:205] Tried to move back new op lower bound from 7317271717663674368 to 7317271717594255360. Current Snapshot: MvccSnapshot[applied={T|T < 7317271717663674368}]
I20260811 10:14:50.451407 854 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:50.451493 854 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:14:50.452258 854 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Sending DeleteTablet for 3 replicas of tablet 04809324126645e79cd204957b7bea7b
I20260811 10:14:50.452584 1067 tablet_service.cc:1514] Processing DeleteTablet for tablet 04809324126645e79cd204957b7bea7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:50 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:34050
I20260811 10:14:50.452580 1289 tablet_service.cc:1514] Processing DeleteTablet for tablet 04809324126645e79cd204957b7bea7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:50 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:50712
I20260811 10:14:50.452601 1529 tablet_service.cc:1514] Processing DeleteTablet for tablet 04809324126645e79cd204957b7bea7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:14:50 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:38480
I20260811 10:14:50.452786 1186 tablet_replica.cc:330] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: stopping tablet replica
I20260811 10:14:50.452805 1680 tablet_replica.cc:330] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: stopping tablet replica
I20260811 10:14:50.452862 1186 raft_consensus.cc:2229] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:50.452893 1680 raft_consensus.cc:2229] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:14:50.452929 1186 raft_consensus.cc:2258] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:50.452943 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 922
I20260811 10:14:50.452979 1680 raft_consensus.cc:2258] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:50.453042 1679 tablet_replica.cc:330] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: stopping tablet replica
I20260811 10:14:50.453128 1679 raft_consensus.cc:2229] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:14:50.453214 1679 raft_consensus.cc:2258] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:14:50.460258 1680 ts_tablet_manager.cc:1905] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:14:50.462670 1680 ts_tablet_manager.cc:1918] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:50.462743 1680 log.cc:1196] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/04809324126645e79cd204957b7bea7b
I20260811 10:14:50.462989 1680 ts_tablet_manager.cc:1939] T 04809324126645e79cd204957b7bea7b P 0cf77aa1b428469c8645d3a1e7679a1c: Deleting consensus metadata
I20260811 10:14:50.465363 834 catalog_manager.cc:4841] TS 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): tablet 04809324126645e79cd204957b7bea7b (table test-table [id=dbb3def92a8443ffb713c8bcae469489]) successfully deleted
I20260811 10:14:50.475804 1679 ts_tablet_manager.cc:1905] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:14:50.478137 1679 ts_tablet_manager.cc:1918] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:14:50.478246 1679 log.cc:1196] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/04809324126645e79cd204957b7bea7b
I20260811 10:14:50.478567 1679 ts_tablet_manager.cc:1939] T 04809324126645e79cd204957b7bea7b P d163f91f51a444ad9944cb1f54e26b33: Deleting consensus metadata
I20260811 10:14:50.479140 834 catalog_manager.cc:4841] TS d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): tablet 04809324126645e79cd204957b7bea7b (table test-table [id=dbb3def92a8443ffb713c8bcae469489]) successfully deleted
W20260811 10:14:50.482753 832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:40427 (error 108)
I20260811 10:14:50.483254 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 1177
I20260811 10:14:50.483349 1181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.483207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:40427 (local address 127.0.0.1:34056)
0811 10:14:50.483259 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.483306 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:40427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 10:14:50.488713 832 catalog_manager.cc:4584] TS a6c882d261c8465592b28bc255cecd51 (127.8.238.1:40427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 04809324126645e79cd204957b7bea7b: Network error: Client connection negotiation failed: client connection to 127.8.238.1:40427: connect: Connection refused (error 111)
I20260811 10:14:50.511416 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 1407
I20260811 10:14:50.521950 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 806
I20260811 10:14:50.532788 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--webserver_interface=127.8.238.62
--webserver_port=35565
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:50.618168 1682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:50.618413 1682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:50.618453 1682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:50.619582 1682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:14:50.619647 1682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:50.619670 1682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:50.619704 1682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:14:50.619767 1682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:14:50.621238 1682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=35565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:50.621513 1682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:50.621743 1682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:50.626710 1691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.626791 1693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.626935 1682 server_base.cc:1034] running on GCE node
W20260811 10:14:50.626786 1690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.627290 1682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:50.627601 1682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:50.628760 1682 hybrid_clock.cc:648] HybridClock initialized: now 1786443290628750 us; error 38 us; skew 500 ppm
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:50.631258 1682 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:14:50.631795 1682 webserver.cc:460] Webserver started at http://127.8.238.62:35565/ using document root <none> and password file <none>
I20260811 10:14:50.632026 1682 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:50.632095 1682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:50.633415 1682 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:50.634122 1704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.634291 1682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:50.634407 1682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e1fcc1d4d03e4264a72515238fb05265"
format_stamp: "Formatted at 2026-08-11 10:14:48 on dist-test-slave-7f01"
server_key: "eeb89f362d4ad7ac2455c6e94a9c7627"
server_key_iv: "74be3b618f57feb84465f15d49dc9305"
server_key_version: "encryptionkey@0"
I20260811 10:14:50.634758 1682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:50.653290 1682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:50.655249 1682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:50.655407 1682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:50.659803 1682 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39947
I20260811 10:14:50.659828 1788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39947 every 8 connection(s)
I20260811 10:14:50.660226 1682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:14:50.661836 1789 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:14:50.662463 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap starting.
I20260811 10:14:50.668505 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 1682
I20260811 10:14:50.668900 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:40427
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=41427
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:50.673285 1789 log.cc:824] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:50.675830 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:14:50.676136 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265: Bootstrap complete.
I20260811 10:14:50.678472 1789 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:50.678694 1789 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.678866 1789 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:50.678937 1789 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:14:50.678983 1789 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:14:50.679034 1789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:14:50.680110 1789 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:50.680200 1789 leader_election.cc:302] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1fcc1d4d03e4264a72515238fb05265; no voters:
I20260811 10:14:50.680418 1789 leader_election.cc:288] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 10:14:50.680461 1796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 10:14:50.680588 1796 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [term 3 LEADER]: Becoming Leader. State: Replica: e1fcc1d4d03e4264a72515238fb05265, State: Running, Role: LEADER
I20260811 10:14:50.680677 1796 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } }
I20260811 10:14:50.680708 1789 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:14:50.680959 1797 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:50.681113 1797 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:50.681411 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1fcc1d4d03e4264a72515238fb05265. Latest consensus state: current_term: 3 leader_uuid: "e1fcc1d4d03e4264a72515238fb05265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1fcc1d4d03e4264a72515238fb05265" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39947 } } }
I20260811 10:14:50.681464 1803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:14:50.681520 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1fcc1d4d03e4264a72515238fb05265 [sys.catalog]: This master's current role is: LEADER
I20260811 10:14:50.682317 1803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:14:50.682458 1803 catalog_manager.cc:1259] Loaded cluster ID: 8a60ef7cae434cfb8dc9a9380d9f1fa7
I20260811 10:14:50.682587 1803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 10:14:50.682974 1803 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f894a5b99ad google::LogMessage::Fail() at ??:0
@ 0x7f894a5bc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f894a5b94de google::LogMessage::Flush() at ??:0
@ 0x7f894a5baa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f89524de0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f89524e78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f894c4870d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f894c47c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f894d2e06db start_thread at ??:0
@ 0x7f89490b761f clone at ??:0
W20260811 10:14:50.769181 1794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:50.769497 1794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:50.769538 1794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:50.770785 1794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:50.770846 1794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:50.770892 1794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:14:50.772449 1794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:40427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=41427
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:50.772694 1794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:50.772943 1794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:50.779131 1862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.779094 1858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.779371 1856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.779419 1794 server_base.cc:1034] running on GCE node
I20260811 10:14:50.779640 1794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:50.779965 1794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:50.781159 1794 hybrid_clock.cc:648] HybridClock initialized: now 1786443290781142 us; error 41 us; skew 500 ppm
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:50.784544 1794 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:14:50.785015 1794 webserver.cc:460] Webserver started at http://127.8.238.1:41427/ using document root <none> and password file <none>
I20260811 10:14:50.785279 1794 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:50.785324 1794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:50.786700 1794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:50.787452 1873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.787671 1794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:50.787808 1794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "a6c882d261c8465592b28bc255cecd51"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "999b47b215004211e86e50094cedf3ed"
server_key_iv: "61e443e1c50d95d452a5ef525fed4cf1"
server_key_version: "encryptionkey@0"
I20260811 10:14:50.788156 1794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:50.806275 1794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:50.808663 1794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:50.808828 1794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:50.809060 1794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:50.809528 1882 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:14:50.811563 1794 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:14:50.811640 1794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:14:50.811688 1794 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:14:50.812407 1794 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:14:50.812462 1794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:14:50.812531 1882 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Bootstrap starting.
I20260811 10:14:50.820127 1794 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:40427
I20260811 10:14:50.820508 1794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:14:50.821023 2050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:40427 every 8 connection(s)
I20260811 10:14:50.824805 1882 log.cc:824] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Log is configured to *not* fsync() on all Append() calls
I20260811 10:14:50.825636 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 1794
I20260811 10:14:50.825896 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:40733
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=40633
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:14:50.827639 1882 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:14:50.828006 1882 tablet_bootstrap.cc:492] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Bootstrap complete.
I20260811 10:14:50.829388 1882 ts_tablet_manager.cc:1397] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.015s
I20260811 10:14:50.830874 1882 raft_consensus.cc:348] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.831223 1882 raft_consensus.cc:729] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c882d261c8465592b28bc255cecd51, State: Initialized, Role: FOLLOWER
I20260811 10:14:50.831414 1882 consensus_queue.cc:260] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:50.831705 1882 ts_tablet_manager.cc:1428] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
W20260811 10:14:50.875701 2054 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:14:50.821383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:42295)
0811 10:14:50.821567 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.821611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:50.822075 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:50.875483 (+ 53408us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
W20260811 10:14:50.875981 2051 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:39947 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:39947: Client connection negotiation failed: client connection to 127.8.238.62:39947: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:14:50.876333 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.876207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:51827)
0811 10:14:50.876265 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.876309 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:14:50.876665 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.876585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:55859)
0811 10:14:50.876622 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.876648 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 10:14:50.876812 2051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:14:50.913332 2056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:50.913548 2056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:50.913571 2056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:50.914721 2056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:50.914763 2056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:50.914788 2056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:14:50.916286 2056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:40733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=40633
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:50.916535 2056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:50.916766 2056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:14:50.921916 2056 server_base.cc:1034] running on GCE node
W20260811 10:14:50.921875 2064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.921839 2063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:50.921871 2069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:50.922329 2056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:50.922573 2056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:50.923713 2056 hybrid_clock.cc:648] HybridClock initialized: now 1786443290923701 us; error 33 us; skew 500 ppm
Aug 11 10:14:50 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:50.926285 2056 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:14:50.926672 2056 webserver.cc:460] Webserver started at http://127.8.238.2:40633/ using document root <none> and password file <none>
I20260811 10:14:50.927102 2056 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:50.927250 2056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:50.928762 2056 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:50.929363 2081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.929535 2056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:14:50.929621 2056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "d163f91f51a444ad9944cb1f54e26b33"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "11db1f0d632f9adce65e14a798bd7044"
server_key_iv: "cff94d9055d4aed4a4f536cd57527902"
server_key_version: "encryptionkey@0"
I20260811 10:14:50.929955 2056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:50.956887 2056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:50.958974 2056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:50.959101 2056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:50.959295 2056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:50.959561 2056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:50.959592 2056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.959614 2056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:50.959627 2056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:50.965909 2056 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:40733
I20260811 10:14:50.965976 2277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:40733 every 8 connection(s)
I20260811 10:14:50.966336 2056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:14:50.967404 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.967021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:53711)
0811 10:14:50.967184 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.967255 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
W20260811 10:14:50.967864 2278 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:39947 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:39947: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
I20260811 10:14:50.968154 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.968050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:42609)
0811 10:14:50.968103 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.968129 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:14:50.968434 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:50.968347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:41465)
0811 10:14:50.968389 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:50.968412 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 10:14:50.968544 2278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:14:50.972203 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 2056
I20260811 10:14:50.972425 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:33823
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:39947
--webserver_port=37113
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:14:51.055917 2290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:14:51.056157 2290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:14:51.056196 2290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:14:51.057304 2290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:14:51.057376 2290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:14:51.057420 2290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:14:51.058825 2290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:33823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=37113
--tserver_master_addrs=127.8.238.62:39947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:14:51.059080 2290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:14:51.059301 2290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:14:51.064242 2298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:51.064260 2308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:14:51.064265 2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:14:51.064515 2290 server_base.cc:1034] running on GCE node
I20260811 10:14:51.064777 2290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:14:51.065006 2290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:14:51.066164 2290 hybrid_clock.cc:648] HybridClock initialized: now 1786443291066152 us; error 27 us; skew 500 ppm
Aug 11 10:14:51 dist-test-slave-7f01 krb5kdc[32422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:14:51.068778 2290 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:14:51.069247 2290 webserver.cc:460] Webserver started at http://127.8.238.3:37113/ using document root <none> and password file <none>
I20260811 10:14:51.069479 2290 fs_manager.cc:359] Metadata directory not provided
I20260811 10:14:51.069545 2290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:14:51.070907 2290 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:51.071544 2319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:51.071724 2290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:14:51.071868 2290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "0cf77aa1b428469c8645d3a1e7679a1c"
format_stamp: "Formatted at 2026-08-11 10:14:49 on dist-test-slave-7f01"
server_key: "b1c1371e581d0c73cd187019cc452233"
server_key_iv: "e6845de10e85c1135dccd0c12b2115b1"
server_key_version: "encryptionkey@0"
I20260811 10:14:51.072208 2290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:14:51.094332 2290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:14:51.096365 2290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:14:51.096531 2290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:14:51.096783 2290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:14:51.097101 2290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:14:51.097155 2290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:51.097198 2290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:14:51.097227 2290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:14:51.103505 2290 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:33823
I20260811 10:14:51.103559 2537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:33823 every 8 connection(s)
I20260811 10:14:51.103961 2290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:14:51.105015 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:51.104640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:53701)
0811 10:14:51.104794 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:51.104868 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
W20260811 10:14:51.105424 2539 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:39947 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:39947: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
I20260811 10:14:51.105695 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:51.105609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:36563)
0811 10:14:51.105651 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:51.105678 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 10:14:51.105978 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:51.105906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:34291)
0811 10:14:51.105941 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:51.105962 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 10:14:51.106086 2539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:14:51.108652 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 2290
I20260811 10:14:51.877511 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:51.877190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:42589)
0811 10:14:51.877375 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:51.877427 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260811 10:14:51.969053 2547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:51.968850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:56157)
0811 10:14:51.968962 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:51.968996 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260811 10:14:52.106688 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.106467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:44633)
0811 10:14:52.106601 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:52.106635 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260811 10:14:52.631166 2550 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:52.631301 2550 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:52.631701 2550 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
Aug 11 10:14:52 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.3@KRBTEST.COM
Aug 11 10:14:52 dist-test-slave-7f01 krb5kdc[32422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443290, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.2@KRBTEST.COM
I20260811 10:14:52.643083 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.632290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:52887 (local address 127.8.238.3:33823)
0811 10:14:52.632437 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:52.632449 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:52.632471 (+ 22us) server_negotiation.cc:407] Connection header received
0811 10:14:52.633025 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:52.633029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:52.633077 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:52.633219 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:52.633761 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.634731 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.635794 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.636208 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.636288 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:52.638757 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:52.638762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:52.638762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:52.638778 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:52.640847 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641486 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:52.641490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:52.641490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:52.641630 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641950 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:52.641951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:52.641952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:52.642286 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:52.642381 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:52.642814 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 10:14:52.642968 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":91,"threads_started":1}
I20260811 10:14:52.643083 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.632013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:58047 (local address 127.8.238.2:40733)
0811 10:14:52.632205 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 10:14:52.632219 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 10:14:52.632246 (+ 27us) server_negotiation.cc:407] Connection header received
0811 10:14:52.632717 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:14:52.632719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:14:52.632760 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:14:52.632887 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:14:52.633995 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.635001 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.635815 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.636223 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.636411 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:52.639051 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:14:52.639055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:14:52.639055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:14:52.639068 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:14:52.641014 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641496 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:52.641499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:52.641499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:52.641630 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641950 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:14:52.641951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:14:52.641952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:14:52.642286 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:14:52.642381 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:14:52.642814 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 10:14:52.642967 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260811 10:14:52.643075 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.631921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:40733 (local address 127.8.238.1:58047)
0811 10:14:52.632076 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:52.632087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:52.632145 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:52.632963 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:52.632966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:52.633427 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:52.633639 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:52.633646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.635093 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.635094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:52.635688 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:52.635690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.636367 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.636369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:52.636456 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:52.637105 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:52.637110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:52.638940 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:52.641083 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:52.641085 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:52.641420 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:52.641706 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:52.641707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:52.641868 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:52.642397 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:52.642408 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:52.642576 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 10:14:52.642707 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 10:14:52.642918 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":24,"thread_start_us":107,"threads_started":1}
I20260811 10:14:52.643131 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.632154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:33823 (local address 127.8.238.1:52887)
0811 10:14:52.632240 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:52.632249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:14:52.632292 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:14:52.633228 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:14:52.633230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:14:52.633444 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:14:52.633639 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:52.633646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.634831 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.634832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:52.635687 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:14:52.635690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:14:52.636327 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:14:52.636328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:14:52.636453 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:14:52.637049 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:14:52.637070 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:14:52.638636 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:14:52.640933 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:52.640940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:52.640941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:52.641378 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:52.641706 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:14:52.641707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:14:52.641707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:14:52.641868 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:14:52.642398 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:14:52.642408 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:14:52.642576 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 10:14:52.642707 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 10:14:52.642976 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":147,"thread_start_us":41,"threads_started":1}
W20260811 10:14:52.644327 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
W20260811 10:14:52.644440 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
I20260811 10:14:52.644479 1877 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6c882d261c8465592b28bc255cecd51; no voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33
I20260811 10:14:52.644593 2550 raft_consensus.cc:2726] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:14:52.878134 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.877971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:56021)
0811 10:14:52.878042 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:52.878081 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 10:14:52.969640 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:52.969429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:43459)
0811 10:14:52.969541 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:52.969587 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260811 10:14:53.107388 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:53.107122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:50935)
0811 10:14:53.107282 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:53.107337 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260811 10:14:53.878862 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:53.878624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:56369)
0811 10:14:53.878776 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:53.878808 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260811 10:14:53.970299 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:53.970090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:46655)
0811 10:14:53.970218 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:53.970252 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260811 10:14:54.108120 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:54.107902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:56343)
0811 10:14:54.108037 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:54.108074 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260811 10:14:54.441332 2572 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:54.441452 2572 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:54.441651 2572 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
W20260811 10:14:54.442111 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
W20260811 10:14:54.442206 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
I20260811 10:14:54.442239 1877 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6c882d261c8465592b28bc255cecd51; no voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33
I20260811 10:14:54.442325 2572 raft_consensus.cc:2726] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:14:54.879550 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:54.879302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:52411)
0811 10:14:54.879458 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:54.879498 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260811 10:14:54.971004 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:54.970741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:35257)
0811 10:14:54.970899 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:54.970938 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
I20260811 10:14:55.108783 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:55.108586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:43047)
0811 10:14:55.108704 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:55.108738 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260811 10:14:55.880435 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:55.880134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:49849)
0811 10:14:55.880324 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:55.880367 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260811 10:14:55.971812 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:55.971509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:33783)
0811 10:14:55.971663 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:55.971704 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260811 10:14:56.109542 2579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:56.109331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:38597)
0811 10:14:56.109457 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:56.109497 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260811 10:14:56.160645 2580 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:56.160758 2580 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:56.160974 2580 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
W20260811 10:14:56.161468 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
W20260811 10:14:56.161535 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
I20260811 10:14:56.161550 1877 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6c882d261c8465592b28bc255cecd51; no voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33
I20260811 10:14:56.161621 2580 raft_consensus.cc:2726] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:14:56.881245 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:56.880980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:47497)
0811 10:14:56.881139 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:56.881180 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260811 10:14:56.972594 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:56.972369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:44909)
0811 10:14:56.972513 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:56.972550 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260811 10:14:57.110258 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:57.110030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:58317)
0811 10:14:57.110173 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:57.110210 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260811 10:14:57.882041 2592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:57.881713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:56291)
0811 10:14:57.881902 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:57.881971 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260811 10:14:57.973235 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:57.973020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:50759)
0811 10:14:57.973151 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:57.973187 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260811 10:14:58.111028 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:58.110800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:52353)
0811 10:14:58.110948 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:58.110983 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260811 10:14:58.882756 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:58.882534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:40711)
0811 10:14:58.882663 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:58.882700 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260811 10:14:58.974062 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:58.973802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:51471)
0811 10:14:58.973970 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:58.974010 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260811 10:14:59.065521 2608 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:14:59.065651 2608 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:14:59.065888 2608 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
W20260811 10:14:59.066385 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
W20260811 10:14:59.066474 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
I20260811 10:14:59.066509 1877 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6c882d261c8465592b28bc255cecd51; no voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33
I20260811 10:14:59.066633 2608 raft_consensus.cc:2726] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:14:59.111864 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:59.111562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:48717)
0811 10:14:59.111713 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:59.111815 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260811 10:14:59.883667 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:59.883349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:40171)
0811 10:14:59.883523 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:59.883580 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260811 10:14:59.974793 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:14:59.974577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:43295)
0811 10:14:59.974709 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 10:14:59.974742 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260811 10:15:00.112664 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:00.112397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:36407)
0811 10:15:00.112565 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:00.112615 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260811 10:15:00.884490 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:00.884231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:36969)
0811 10:15:00.884387 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:00.884435 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260811 10:15:00.975663 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:00.975352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:46753)
0811 10:15:00.975559 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:00.975612 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
I20260811 10:15:01.113406 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:01.113130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:57041)
0811 10:15:01.113320 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:01.113360 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":65,"threads_started":1}
I20260811 10:15:01.885278 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:01.885007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:39323)
0811 10:15:01.885145 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:01.885205 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260811 10:15:01.976478 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:01.976263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:42909)
0811 10:15:01.976390 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:01.976427 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260811 10:15:02.114195 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:02.113927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:39381)
0811 10:15:02.114094 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:02.114141 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260811 10:15:02.886090 2626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:02.885851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:38231)
0811 10:15:02.885994 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:02.886032 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260811 10:15:02.977197 2628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:02.976946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:36363)
0811 10:15:02.977111 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:02.977149 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260811 10:15:03.114985 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:03.114742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:39479)
0811 10:15:03.114894 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:03.114934 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
I20260811 10:15:03.886845 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:03.886593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:43939)
0811 10:15:03.886736 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:03.886789 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260811 10:15:03.977950 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:03.977709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:53393)
0811 10:15:03.977865 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:03.977900 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260811 10:15:04.115785 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:04.115511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:55033)
0811 10:15:04.115653 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:04.115690 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260811 10:15:04.887558 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:04.887299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:34685)
0811 10:15:04.887464 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:04.887502 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260811 10:15:04.978721 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:04.978493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:48311)
0811 10:15:04.978635 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:04.978671 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260811 10:15:05.116497 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:05.116269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:58545)
0811 10:15:05.116416 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:05.116451 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260811 10:15:05.888394 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:05.888164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.1:53353)
0811 10:15:05.888301 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:05.888340 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260811 10:15:05.979501 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:05.979266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.2:39097)
0811 10:15:05.979412 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:05.979452 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260811 10:15:05.982254 2645 raft_consensus.cc:482] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:05.982369 2645 raft_consensus.cc:504] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d163f91f51a444ad9944cb1f54e26b33" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 40733 } } peers { permanent_uuid: "0cf77aa1b428469c8645d3a1e7679a1c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 33823 } } peers { permanent_uuid: "a6c882d261c8465592b28bc255cecd51" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 40427 } }
I20260811 10:15:05.982592 2645 leader_election.cc:288] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733), 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823)
W20260811 10:15:05.983008 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0cf77aa1b428469c8645d3a1e7679a1c (127.8.238.3:33823): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
W20260811 10:15:05.983088 1877 leader_election.cc:341] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d163f91f51a444ad9944cb1f54e26b33 (127.8.238.2:40733): Not found: Tablet not found: 04809324126645e79cd204957b7bea7b
I20260811 10:15:05.983103 1877 leader_election.cc:302] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6c882d261c8465592b28bc255cecd51; no voters: 0cf77aa1b428469c8645d3a1e7679a1c, d163f91f51a444ad9944cb1f54e26b33
I20260811 10:15:05.983192 2645 raft_consensus.cc:2726] T 04809324126645e79cd204957b7bea7b P a6c882d261c8465592b28bc255cecd51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:15:06.116923 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 1794
I20260811 10:15:06.124058 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:06.117147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39947 (local address 127.8.238.3:33461)
0811 10:15:06.123939 (+ 6792us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:06.123990 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:39947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6767,"thread_start_us":58,"threads_started":1}
I20260811 10:15:06.133595 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 2056
I20260811 10:15:06.154920 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 2290
2026-08-11T10:15:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18690 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:15:06 dist-test-slave-7f01 krb5kdc[2656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:15:06 dist-test-slave-7f01 krb5kdc[2656](info): set up 2 sockets
Aug 11 10:15:06 dist-test-slave-7f01 krb5kdc[2656](info): commencing operation
krb5kdc: starting...
W20260811 10:15:07.440793 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:15:07 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:15:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:15:07.567447 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:07.649765 2682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:07.649981 2682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:07.650002 2682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:07.651058 2682 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:15:07.651090 2682 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:15:07.651103 2682 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:15:07.651115 2682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:15:07.651129 2682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:07.651141 2682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:07.651153 2682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:15:07.651175 2682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:15:07.652572 2682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:07.652758 2682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:07.652937 2682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:07.658020 2691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:07.658092 2690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:07.658209 2682 server_base.cc:1034] running on GCE node
W20260811 10:15:07.658092 2693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:07.658493 2682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:07.658777 2682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:07.659965 2682 hybrid_clock.cc:648] HybridClock initialized: now 1786443307659958 us; error 38 us; skew 500 ppm
Aug 11 10:15:07 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443307, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:07.663244 2682 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:15:07.663635 2682 webserver.cc:460] Webserver started at http://127.8.238.62:42757/ using document root <none> and password file <none>
I20260811 10:15:07.663882 2682 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:07.663935 2682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:07.664023 2682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:15:07.665004 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "db3faf0c95b5411e919736cc1ef80a31"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "44ed8aa508ef2755a2b3dd668500a972"
server_key_iv: "4ce51635781407bb25295642103fe491"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.665308 2682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "db3faf0c95b5411e919736cc1ef80a31"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "44ed8aa508ef2755a2b3dd668500a972"
server_key_iv: "4ce51635781407bb25295642103fe491"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.666587 2682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:15:07.667266 2707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:07.667459 2682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:15:07.667557 2682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "db3faf0c95b5411e919736cc1ef80a31"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "44ed8aa508ef2755a2b3dd668500a972"
server_key_iv: "4ce51635781407bb25295642103fe491"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.667630 2682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:07.678728 2682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:07.681043 2682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:07.681202 2682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:07.685648 2682 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:41953
I20260811 10:15:07.685686 2803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:41953 every 8 connection(s)
I20260811 10:15:07.686072 2682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:15:07.686951 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:07.689400 2804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap starting.
I20260811 10:15:07.690022 2804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:07.690358 2804 log.cc:824] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:07.691056 2804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: No bootstrap required, opened a new log
I20260811 10:15:07.692782 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 2682
I20260811 10:15:07.692909 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:15:07.692776 2804 raft_consensus.cc:348] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:07.692945 2804 raft_consensus.cc:374] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:07.693022 2804 raft_consensus.cc:729] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Initialized, Role: FOLLOWER
I20260811 10:15:07.693143 9144 external_mini_cluster.cc:1428] Setting key 6ec7a08f22c50d7f8899f74caf2a8358
I20260811 10:15:07.693179 2804 consensus_queue.cc:260] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:07.693259 2804 raft_consensus.cc:388] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:15:07.693282 2804 raft_consensus.cc:482] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:15:07.693312 2804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:07.694336 2804 raft_consensus.cc:504] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:07.694432 2804 leader_election.cc:302] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db3faf0c95b5411e919736cc1ef80a31; no voters:
I20260811 10:15:07.694633 2804 leader_election.cc:288] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:15:07.694669 2816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:15:07.694784 2816 raft_consensus.cc:686] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 LEADER]: Becoming Leader. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Running, Role: LEADER
I20260811 10:15:07.694917 2804 sys_catalog.cc:564] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:15:07.694931 2816 consensus_queue.cc:237] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:07.695465 2818 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db3faf0c95b5411e919736cc1ef80a31. Latest consensus state: current_term: 1 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:07.695592 2818 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:07.695853 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:07.695972 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:07.696159 2827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:15:07.696728 2827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:15:07.698381 2827 catalog_manager.cc:1347] Generated new cluster ID: 764bdeb443234f0584d01a1ac53297a0
I20260811 10:15:07.698467 2827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:15:07 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:07.705133 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:07.693672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.8.238.62:41953)
0811 10:15:07.693881 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:07.693890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:07.693921 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:15:07.694025 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:07.694030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:07.694069 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:07.694202 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:07.695010 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.696462 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.697667 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.698271 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.698407 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:07.701298 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:07.701303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:07.701303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:07.701315 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:07.703414 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:07.703835 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:07.703838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:07.703838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:07.703974 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:07.704187 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:07.704188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:07.704188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:07.704519 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:07.704683 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:07.704768 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 10:15:07.704921 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260811 10:15:07.714210 2827 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:15:07.714759 2827 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:15:07.720088 2827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Generated new TSK 0
I20260811 10:15:07.720285 2827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:15:07.751325 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:07.857550 2848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:07.857788 2848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:07.857827 2848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:07.858982 2848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:07.859046 2848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:07.859092 2848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:15:07.860605 2848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:07.860862 2848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:07.861083 2848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:07.866158 2856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:07.866169 2859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:07.866158 2857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:07.866174 2848 server_base.cc:1034] running on GCE node
I20260811 10:15:07.866580 2848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:07.866863 2848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:07.868029 2848 hybrid_clock.cc:648] HybridClock initialized: now 1786443307868011 us; error 39 us; skew 500 ppm
Aug 11 10:15:07 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443307, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:07.870714 2848 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:15:07.871196 2848 webserver.cc:460] Webserver started at http://127.8.238.1:38691/ using document root <none> and password file <none>
I20260811 10:15:07.871492 2848 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:07.871567 2848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:07.871768 2848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:15:07.872848 2848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "ff9f7b37be68452bb83f552c26e68d61"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "01208b288a7c0c0f21885cf76159f65c"
server_key_iv: "f98d4746a822da73c0efb1102db3d6c1"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.873235 2848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "ff9f7b37be68452bb83f552c26e68d61"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "01208b288a7c0c0f21885cf76159f65c"
server_key_iv: "f98d4746a822da73c0efb1102db3d6c1"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.874590 2848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:15:07.875319 2868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:07.875502 2848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 10:15:07.875618 2848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "ff9f7b37be68452bb83f552c26e68d61"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "01208b288a7c0c0f21885cf76159f65c"
server_key_iv: "f98d4746a822da73c0efb1102db3d6c1"
server_key_version: "encryptionkey@0"
I20260811 10:15:07.875697 2848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:07.902599 2848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:07.904809 2848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:07.904999 2848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:07.905278 2848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:07.905627 2848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:07.905684 2848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:07.905726 2848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:07.905772 2848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:07.911851 2848 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:41307
I20260811 10:15:07.911957 3064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:41307 every 8 connection(s)
I20260811 10:15:07.912299 2848 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:15:07.917807 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 2848
I20260811 10:15:07.917939 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:15:07.918161 9144 external_mini_cluster.cc:1428] Setting key 2b0aa102a05626250ba276dd4b73dc76
Aug 11 10:15:07 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443307, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:07.923045 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:07.913155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:59585 (local address 127.8.238.62:41953)
0811 10:15:07.913271 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:07.913273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:07.913466 (+ 193us) server_negotiation.cc:407] Connection header received
0811 10:15:07.914047 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:07.914048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:07.914066 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:07.914130 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:07.915157 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.915686 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.916735 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.916960 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.917041 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:07.919683 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:07.919685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:07.919686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:07.919697 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:07.921389 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:07.921934 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:07.921935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:07.921935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:07.921989 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:07.922288 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:07.922288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:07.922289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:07.922392 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:07.922452 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:07.922823 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 10:15:07.922950 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 10:15:07.923187 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:07.913044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:59585)
0811 10:15:07.913314 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:07.913369 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:07.913931 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:07.914215 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:07.914219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:07.914837 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:07.915045 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:07.915051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.915811 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.915812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:07.916640 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:07.916642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:07.917049 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:07.917050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:07.917183 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:07.917688 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:07.917699 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:07.919591 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:07.921490 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:07.921499 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:07.921502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:07.921846 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:07.922054 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:07.922055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:07.922055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:07.922208 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:07.922456 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:07.922463 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:07.922632 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 10:15:07.922778 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 10:15:07.922953 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":123,"threads_started":1}
I20260811 10:15:07.924126 3067 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:07.924260 3067 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:07.924482 3067 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:07.925143 2738 ts_manager.cc:194] Registered new tserver with Master: ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307)
I20260811 10:15:07.926141 2738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:59585
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:15:07.948585 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:08.032804 3075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:08.033010 3075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:08.033034 3075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:08.034169 3075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:08.034211 3075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:08.034237 3075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:15:08.035676 3075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:08.035923 3075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:08.036109 3075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:08.041044 3085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:08.041111 3082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:08.041131 3075 server_base.cc:1034] running on GCE node
W20260811 10:15:08.041046 3081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:08.041510 3075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:08.041754 3075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:08.042903 3075 hybrid_clock.cc:648] HybridClock initialized: now 1786443308042850 us; error 77 us; skew 500 ppm
Aug 11 10:15:08 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443308, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:08.045580 3075 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:15:08.046058 3075 webserver.cc:460] Webserver started at http://127.8.238.2:46353/ using document root <none> and password file <none>
I20260811 10:15:08.046283 3075 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:08.046350 3075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:08.046517 3075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:15:08.047545 3075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "4424687e685b42908b51fb7bb0b4a4b2"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "2b70794c8e14f771a30e5d74c322a583"
server_key_iv: "05773ba64a54866030b59d41fae00e90"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.047940 3075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "4424687e685b42908b51fb7bb0b4a4b2"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "2b70794c8e14f771a30e5d74c322a583"
server_key_iv: "05773ba64a54866030b59d41fae00e90"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.049273 3075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:08.050004 3099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.050191 3075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:15:08.050300 3075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "4424687e685b42908b51fb7bb0b4a4b2"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "2b70794c8e14f771a30e5d74c322a583"
server_key_iv: "05773ba64a54866030b59d41fae00e90"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.050374 3075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:08.065622 3075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:08.067445 3075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:08.067608 3075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:08.067886 3075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:08.068203 3075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:08.068262 3075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.068305 3075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:08.068332 3075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.074731 3075 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:34501
I20260811 10:15:08.074786 3295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:34501 every 8 connection(s)
I20260811 10:15:08.075095 3075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:15:08 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443308, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:08.084263 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 3075
I20260811 10:15:08.084375 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:15:08.084479 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.075860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:52117 (local address 127.8.238.62:41953)
0811 10:15:08.075913 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.075915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.076167 (+ 252us) server_negotiation.cc:407] Connection header received
0811 10:15:08.076728 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.076729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.076743 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.076789 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:08.077755 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.078278 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.079338 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.079547 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.079656 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.081848 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:08.081850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:08.081850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:08.081856 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:08.083081 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.083475 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.083476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.083476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.083532 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.083797 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.083798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.083798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.083910 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:08.083967 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.084236 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.084340 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:15:08.084580 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.075865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:52117)
0811 10:15:08.076046 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.076082 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.076612 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.076867 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.076871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.077459 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:08.077647 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.077652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.078379 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.078380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.079249 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.079251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.079666 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.079668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.079864 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.080371 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:08.080380 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:08.081764 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:08.083156 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:08.083163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:08.083165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:08.083411 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:08.083596 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:08.083597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:08.083597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:08.083699 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:08.083971 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:08.083976 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:08.084091 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.084165 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.084280 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260811 10:15:08.084628 9144 external_mini_cluster.cc:1428] Setting key 015a5366a43edd5b8924775ee9088fa9
I20260811 10:15:08.085095 3297 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:08.085197 3297 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:08.085369 3297 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:08.085742 2738 ts_manager.cc:194] Registered new tserver with Master: 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501)
I20260811 10:15:08.086112 2738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:52117
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:15:08.112112 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:08.194253 3304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:08.194484 3304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:08.194526 3304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:08.195920 3304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:08.195982 3304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:08.196025 3304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:15:08.197532 3304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:08.197805 3304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:08.198046 3304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:08.203114 3317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:08.203136 3312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:08.203351 3304 server_base.cc:1034] running on GCE node
W20260811 10:15:08.203150 3314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:08.203625 3304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:08.203902 3304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:08.205065 3304 hybrid_clock.cc:648] HybridClock initialized: now 1786443308205052 us; error 28 us; skew 500 ppm
Aug 11 10:15:08 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443308, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:08.207653 3304 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:15:08.208163 3304 webserver.cc:460] Webserver started at http://127.8.238.3:42801/ using document root <none> and password file <none>
I20260811 10:15:08.208410 3304 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:08.208478 3304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:08.208626 3304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:15:08.209620 3304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "60c75a4d543daf6eb109a7b0f04d243a"
server_key_iv: "841e44a788f8f42809945a0cf8aafc7b"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.209997 3304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "60c75a4d543daf6eb109a7b0f04d243a"
server_key_iv: "841e44a788f8f42809945a0cf8aafc7b"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.211297 3304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:15:08.212113 3329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.212297 3304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:15:08.212407 3304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "60c75a4d543daf6eb109a7b0f04d243a"
server_key_iv: "841e44a788f8f42809945a0cf8aafc7b"
server_key_version: "encryptionkey@0"
I20260811 10:15:08.212482 3304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:08.242623 3304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:08.244642 3304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:08.244812 3304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:08.245061 3304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:08.245385 3304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:08.245443 3304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.245487 3304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:08.245534 3304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:08.251706 3304 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43881
I20260811 10:15:08.251782 3555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43881 every 8 connection(s)
I20260811 10:15:08.252091 3304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:15:08.257058 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 3304
I20260811 10:15:08.257161 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:15:08.257362 9144 external_mini_cluster.cc:1428] Setting key 4aed70677e1785449b238d9ada670e10
Aug 11 10:15:08 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443308, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:08.261519 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.252770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:34125 (local address 127.8.238.62:41953)
0811 10:15:08.252856 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.252858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.253063 (+ 205us) server_negotiation.cc:407] Connection header received
0811 10:15:08.253582 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.253583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.253596 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.253657 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:08.254647 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.255230 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.256273 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.256460 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.256512 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.258934 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:08.258936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:08.258936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:08.258944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:08.260397 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.260744 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.260745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.260745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.260796 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.261021 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.261022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.261022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.261123 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:08.261173 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.261397 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.261454 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 10:15:08.261678 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.252736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:34125)
0811 10:15:08.252938 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.252975 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.253464 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.253764 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.253769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.254365 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:08.254564 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.254569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.255310 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.255311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.256200 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.256203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.256533 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.256533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.256629 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.257086 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:08.257096 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:08.258809 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:08.260464 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:08.260470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:08.260472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:08.260679 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:08.260857 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:08.260858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:08.260858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:08.260967 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:08.261178 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:08.261182 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:08.261289 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.261366 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.261484 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260811 10:15:08.262135 3557 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:08.262218 3557 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:08.262428 3557 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:08.262838 2738 ts_manager.cc:194] Registered new tserver with Master: 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
I20260811 10:15:08.263207 2738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:34125
I20260811 10:15:08.269825 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:15:08.277253 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.271268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.8.238.62:41953)
0811 10:15:08.271340 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.271342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.271445 (+ 103us) server_negotiation.cc:407] Connection header received
0811 10:15:08.271532 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.271533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.271544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.271595 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:08.272235 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.272715 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.273435 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.273649 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.273723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.274843 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:08.274845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:08.274845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:08.274852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:08.276065 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.276411 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.276412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.276413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.276471 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:08.276667 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:08.276668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:08.276669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:08.276818 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:08.276933 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.277074 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.277152 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:15:08.278728 2738 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:15:08.280550 2738 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:15:08.287276 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43881 (local address 127.0.0.1:41726)
0811 10:15:08.283386 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.283395 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.283440 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.283905 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.283909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.283912 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:08.284004 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.284006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.285037 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.285038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.286169 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.286171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.286986 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.286987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.287064 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.287077 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.287151 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.287201 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":105,"threads_started":1}
I20260811 10:15:08.287293 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.8.238.3:43881)
0811 10:15:08.283555 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.283561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.283573 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:15:08.283616 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.283618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.283635 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.283732 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:08.284096 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.284900 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.286263 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.286898 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.286986 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.287048 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.287167 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.287208 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260811 10:15:08.287638 3579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41307 (local address 127.0.0.1:37768)
0811 10:15:08.283524 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.283532 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.283576 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.284005 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.284007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.284009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:08.284069 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.284071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.285447 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.285448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.286459 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.286462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.287405 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.287406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.287479 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.287483 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.287512 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.287551 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260811 10:15:08.288019 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.8.238.1:41307)
0811 10:15:08.283830 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.283834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.283864 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:15:08.283907 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.283909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.283925 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.284010 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:08.284161 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.285339 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.286543 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.287295 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.287787 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.287859 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.287898 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.287943 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260811 10:15:08.288239 3433 tablet_service.cc:1467] Processing CreateTablet for tablet ed66e292ca504547972ee83b4e9ed901 (DEFAULT_TABLE table=test-table [id=62285a9683f641d2a012a7092219a09c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:08.288661 3433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed66e292ca504547972ee83b4e9ed901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:08.288704 2958 tablet_service.cc:1467] Processing CreateTablet for tablet ed66e292ca504547972ee83b4e9ed901 (DEFAULT_TABLE table=test-table [id=62285a9683f641d2a012a7092219a09c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:08.289074 2958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed66e292ca504547972ee83b4e9ed901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:08.291882 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:34501 (local address 127.0.0.1:52394)
0811 10:15:08.283294 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.283338 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.283423 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.284224 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.284225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.284228 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:08.284266 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.284267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.285502 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.285504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.286517 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.286520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.287610 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.287611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.287668 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.287672 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.287704 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.287772 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":55,"threads_started":1}
I20260811 10:15:08.292249 3589 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Bootstrap starting.
I20260811 10:15:08.292340 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.283196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.8.238.2:34501)
0811 10:15:08.283871 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.283875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.283886 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:15:08.283930 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.283932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.284124 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.284212 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:08.284338 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.285423 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.286586 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.287295 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.292074 (+ 4779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.292159 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.292189 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.292244 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":95,"threads_started":1}
I20260811 10:15:08.292588 3590 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap starting.
I20260811 10:15:08.293031 3589 tablet_bootstrap.cc:654] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:08.293251 3590 tablet_bootstrap.cc:654] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:08.293321 3175 tablet_service.cc:1467] Processing CreateTablet for tablet ed66e292ca504547972ee83b4e9ed901 (DEFAULT_TABLE table=test-table [id=62285a9683f641d2a012a7092219a09c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:08.293658 3589 log.cc:824] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:08.293717 3175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed66e292ca504547972ee83b4e9ed901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:08.293876 3590 log.cc:824] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:08.294669 3590 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: No bootstrap required, opened a new log
I20260811 10:15:08.294736 3590 ts_tablet_manager.cc:1397] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:15:08.294941 3589 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: No bootstrap required, opened a new log
I20260811 10:15:08.294998 3589 ts_tablet_manager.cc:1397] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:15:08.296859 3590 raft_consensus.cc:348] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.297067 3590 raft_consensus.cc:374] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:08.296963 3589 raft_consensus.cc:348] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.297077 3594 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Bootstrap starting.
I20260811 10:15:08.297118 3589 raft_consensus.cc:374] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:08.297117 3590 raft_consensus.cc:729] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff9f7b37be68452bb83f552c26e68d61, State: Initialized, Role: FOLLOWER
I20260811 10:15:08.297152 3589 raft_consensus.cc:729] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b9a1fa12ad499fa46b3b652322dd5c, State: Initialized, Role: FOLLOWER
I20260811 10:15:08.297243 3590 consensus_queue.cc:260] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.297268 3589 consensus_queue.cc:260] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.297547 3590 ts_tablet_manager.cc:1428] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:15:08.297546 3589 ts_tablet_manager.cc:1428] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:15:08.297569 3067 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
I20260811 10:15:08.297674 3557 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
I20260811 10:15:08.298065 3594 tablet_bootstrap.cc:654] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:08.298641 3594 log.cc:824] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:08.299417 3594 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: No bootstrap required, opened a new log
I20260811 10:15:08.299474 3594 ts_tablet_manager.cc:1397] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:15:08.300902 3594 raft_consensus.cc:348] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.301043 3594 raft_consensus.cc:374] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:08.301065 3594 raft_consensus.cc:729] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4424687e685b42908b51fb7bb0b4a4b2, State: Initialized, Role: FOLLOWER
I20260811 10:15:08.301208 3594 consensus_queue.cc:260] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.301476 3594 ts_tablet_manager.cc:1428] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:15:08.301529 3297 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
W20260811 10:15:08.325680 3298 tablet.cc:2369] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:15:08.413102 3068 tablet.cc:2369] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:15:08.452911 3600 raft_consensus.cc:482] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:08.453058 3600 raft_consensus.cc:504] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.453518 3600 leader_election.cc:288] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881), ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307)
I20260811 10:15:08.457608 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.453832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:42203 (local address 127.8.238.1:41307)
0811 10:15:08.453928 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.453930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.453942 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:15:08.453982 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.453983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.453997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.454076 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:08.454261 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.455076 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.456602 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.457191 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.457275 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.457287 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.457493 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.457535 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:15:08.457608 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.453876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:46407 (local address 127.8.238.3:43881)
0811 10:15:08.453935 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.453937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.454270 (+ 333us) server_negotiation.cc:407] Connection header received
0811 10:15:08.454309 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.454310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.454322 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.454377 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:08.454536 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.455214 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.456601 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.457191 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.457275 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.457288 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.457493 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.457535 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:15:08.457633 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.453717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41307 (local address 127.8.238.2:42203)
0811 10:15:08.453774 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.453787 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.453839 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.454090 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.454093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.454095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:08.454181 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.454184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.455152 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.455153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.456483 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.456487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.457288 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.457289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.457398 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.457405 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.457466 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.457517 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":30}
I20260811 10:15:08.457639 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.453720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43881 (local address 127.8.238.2:46407)
0811 10:15:08.454160 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.454170 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:08.454228 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:08.454397 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:08.454398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:08.454400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:08.454464 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.454466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.455284 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.455285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.456483 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:08.456487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.457290 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.457291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:08.457398 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.457405 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:15:08.457466 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 10:15:08.457517 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":216,"thread_start_us":163,"threads_started":1}
I20260811 10:15:08.458141 2993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed66e292ca504547972ee83b4e9ed901" candidate_uuid: "4424687e685b42908b51fb7bb0b4a4b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff9f7b37be68452bb83f552c26e68d61" is_pre_election: true
I20260811 10:15:08.458252 3473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed66e292ca504547972ee83b4e9ed901" candidate_uuid: "4424687e685b42908b51fb7bb0b4a4b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" is_pre_election: true
I20260811 10:15:08.458333 2993 raft_consensus.cc:2454] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4424687e685b42908b51fb7bb0b4a4b2 in term 0.
I20260811 10:15:08.458400 3473 raft_consensus.cc:2454] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4424687e685b42908b51fb7bb0b4a4b2 in term 0.
I20260811 10:15:08.458580 3103 leader_election.cc:302] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4424687e685b42908b51fb7bb0b4a4b2, ff9f7b37be68452bb83f552c26e68d61; no voters:
I20260811 10:15:08.458709 3600 raft_consensus.cc:2781] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:15:08.458768 3600 raft_consensus.cc:482] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:15:08.458793 3600 raft_consensus.cc:3037] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:08.459558 3600 raft_consensus.cc:504] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.459702 3600 leader_election.cc:288] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [CANDIDATE]: Term 1 election: Requested vote from peers 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881), ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307)
I20260811 10:15:08.459883 3473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed66e292ca504547972ee83b4e9ed901" candidate_uuid: "4424687e685b42908b51fb7bb0b4a4b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
I20260811 10:15:08.459900 2993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed66e292ca504547972ee83b4e9ed901" candidate_uuid: "4424687e685b42908b51fb7bb0b4a4b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff9f7b37be68452bb83f552c26e68d61"
I20260811 10:15:08.459965 3473 raft_consensus.cc:3037] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:08.460011 2993 raft_consensus.cc:3037] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:08.460755 3473 raft_consensus.cc:2454] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4424687e685b42908b51fb7bb0b4a4b2 in term 1.
I20260811 10:15:08.460784 2993 raft_consensus.cc:2454] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4424687e685b42908b51fb7bb0b4a4b2 in term 1.
I20260811 10:15:08.460975 3103 leader_election.cc:302] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4424687e685b42908b51fb7bb0b4a4b2, ff9f7b37be68452bb83f552c26e68d61; no voters:
I20260811 10:15:08.461133 3600 raft_consensus.cc:2781] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:15:08.461285 3600 raft_consensus.cc:686] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 LEADER]: Becoming Leader. State: Replica: 4424687e685b42908b51fb7bb0b4a4b2, State: Running, Role: LEADER
I20260811 10:15:08.461385 3600 consensus_queue.cc:237] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:08.462335 2737 catalog_manager.cc:5495] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2). New cstate: current_term: 1 leader_uuid: "4424687e685b42908b51fb7bb0b4a4b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } health_report { overall_health: UNKNOWN } } }
W20260811 10:15:08.502650 3558 tablet.cc:2369] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:15:08.513075 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.510668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.8.238.2:34501)
0811 10:15:08.510739 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:08.510741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:08.510752 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:15:08.510795 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:08.510796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:08.510819 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:08.510880 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:15:08.511019 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.511498 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.512289 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:08.512495 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:08.512561 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:08.512710 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:15:08.512825 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:15:08.512892 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:08.512931 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:15:08.512947 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:15:08.515158 3473 raft_consensus.cc:1261] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Refusing update from remote peer 4424687e685b42908b51fb7bb0b4a4b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:15:08.515169 2993 raft_consensus.cc:1261] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Refusing update from remote peer 4424687e685b42908b51fb7bb0b4a4b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:15:08.515563 3600 consensus_queue.cc:1035] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:15:08.515782 3604 consensus_queue.cc:1035] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:15:08.518391 3607 mvcc.cc:205] Tried to move back new op lower bound from 7317271791674691584 to 7317271791458267136. Current Snapshot: MvccSnapshot[applied={T|T < 7317271791674691584}]
I20260811 10:15:08.518582 3610 mvcc.cc:205] Tried to move back new op lower bound from 7317271791674691584 to 7317271791458267136. Current Snapshot: MvccSnapshot[applied={T|T < 7317271791674691584}]
I20260811 10:15:08.520928 2737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:15:08.521018 2737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:15:08.521867 2737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Sending DeleteTablet for 3 replicas of tablet ed66e292ca504547972ee83b4e9ed901
I20260811 10:15:08.522235 3175 tablet_service.cc:1514] Processing DeleteTablet for tablet ed66e292ca504547972ee83b4e9ed901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:08 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:52394
I20260811 10:15:08.522307 3433 tablet_service.cc:1514] Processing DeleteTablet for tablet ed66e292ca504547972ee83b4e9ed901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:08 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:41726
I20260811 10:15:08.522315 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 2848
I20260811 10:15:08.522503 2958 tablet_service.cc:1514] Processing DeleteTablet for tablet ed66e292ca504547972ee83b4e9ed901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:08 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:37768
I20260811 10:15:08.544378 3625 tablet_replica.cc:330] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: stopping tablet replica
I20260811 10:15:08.544543 3625 raft_consensus.cc:2229] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:15:08.544708 3625 raft_consensus.cc:2258] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:15:08.544948 3620 tablet_replica.cc:330] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: stopping tablet replica
I20260811 10:15:08.545064 3620 raft_consensus.cc:2229] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:15:08.545172 3620 raft_consensus.cc:2258] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:15:08.546068 3620 ts_tablet_manager.cc:1905] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:15:08.546200 3625 ts_tablet_manager.cc:1905] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:15:08.548097 3625 ts_tablet_manager.cc:1918] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:15:08.548097 3620 ts_tablet_manager.cc:1918] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:15:08.548182 3620 log.cc:1196] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/ed66e292ca504547972ee83b4e9ed901
I20260811 10:15:08.548295 3625 log.cc:1196] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/ed66e292ca504547972ee83b4e9ed901
I20260811 10:15:08.548549 3620 ts_tablet_manager.cc:1939] T ed66e292ca504547972ee83b4e9ed901 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting consensus metadata
I20260811 10:15:08.548656 3625 ts_tablet_manager.cc:1939] T ed66e292ca504547972ee83b4e9ed901 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting consensus metadata
I20260811 10:15:08.549081 2716 catalog_manager.cc:4841] TS 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): tablet ed66e292ca504547972ee83b4e9ed901 (table test-table [id=62285a9683f641d2a012a7092219a09c]) successfully deleted
I20260811 10:15:08.549144 2708 catalog_manager.cc:4841] TS 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): tablet ed66e292ca504547972ee83b4e9ed901 (table test-table [id=62285a9683f641d2a012a7092219a09c]) successfully deleted
W20260811 10:15:08.550256 2714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:41307 (error 108)
I20260811 10:15:08.550581 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 3075
I20260811 10:15:08.555131 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:08.550704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41307 (local address 127.0.0.1:37782)
0811 10:15:08.555025 (+ 4321us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:08.555085 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2034}
W20260811 10:15:08.556459 2714 catalog_manager.cc:4584] TS ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed66e292ca504547972ee83b4e9ed901: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41307: connect: Connection refused (error 111)
I20260811 10:15:08.563020 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 3304
I20260811 10:15:08.576282 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 2682
I20260811 10:15:28.588405 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--webserver_interface=127.8.238.62
--webserver_port=42757
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:28.672487 3627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:28.672688 3627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:28.672711 3627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:28.673794 3627 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:15:28.673827 3627 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:15:28.673840 3627 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:15:28.673852 3627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:15:28.673866 3627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:28.673878 3627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:28.673890 3627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:15:28.673902 3627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:15:28.675261 3627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:28.675439 3627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:28.675602 3627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:28.685035 3641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:28.685045 3637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:28.685324 3627 server_base.cc:1034] running on GCE node
W20260811 10:15:28.685035 3643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:28.685663 3627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:28.685979 3627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:28.687134 3627 hybrid_clock.cc:648] HybridClock initialized: now 1786443328687111 us; error 37 us; skew 500 ppm
Aug 11 10:15:28 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:28.694576 3627 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:15:28.694977 3627 webserver.cc:460] Webserver started at http://127.8.238.62:42757/ using document root <none> and password file <none>
I20260811 10:15:28.695170 3627 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:28.695225 3627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:28.696681 3627 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:28.697337 3652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:28.697535 3627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:28.697628 3627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "db3faf0c95b5411e919736cc1ef80a31"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "44ed8aa508ef2755a2b3dd668500a972"
server_key_iv: "4ce51635781407bb25295642103fe491"
server_key_version: "encryptionkey@0"
I20260811 10:15:28.698031 3627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:28.718597 3627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:28.720894 3627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:28.721011 3627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:28.725188 3627 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:41953
I20260811 10:15:28.725272 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:41953 every 8 connection(s)
I20260811 10:15:28.725646 3627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:15:28.727243 3765 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:15:28.728004 3765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap starting.
I20260811 10:15:28.735328 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 3627
I20260811 10:15:28.735641 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:41307
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=38691
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:15:28.737915 3765 log.cc:824] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:28.739784 3765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:15:28.740115 3765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap complete.
I20260811 10:15:28.743014 3765 raft_consensus.cc:348] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:28.743360 3765 raft_consensus.cc:729] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Initialized, Role: FOLLOWER
I20260811 10:15:28.743611 3765 consensus_queue.cc:260] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:28.743707 3765 raft_consensus.cc:388] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:15:28.743782 3765 raft_consensus.cc:482] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:15:28.743848 3765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 1 FOLLOWER]: Advancing to term 2
I20260811 10:15:28.745432 3765 raft_consensus.cc:504] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:28.745556 3765 leader_election.cc:302] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db3faf0c95b5411e919736cc1ef80a31; no voters:
I20260811 10:15:28.745747 3765 leader_election.cc:288] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 10:15:28.745790 3770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 10:15:28.745919 3770 raft_consensus.cc:686] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 LEADER]: Becoming Leader. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Running, Role: LEADER
I20260811 10:15:28.746030 3765 sys_catalog.cc:564] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:15:28.746021 3770 consensus_queue.cc:237] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:28.747004 3773 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db3faf0c95b5411e919736cc1ef80a31. Latest consensus state: current_term: 2 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:28.747105 3773 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:28.747301 3784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:15:28.747453 3772 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:28.747522 3772 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:28.748288 3784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:15:28.748457 3784 catalog_manager.cc:1259] Loaded cluster ID: 764bdeb443234f0584d01a1ac53297a0
I20260811 10:15:28.748670 3784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:15:28.749822 3784 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:15:28.750012 3784 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Loaded TSK: 0
I20260811 10:15:28.765909 3784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Generated new TSK 1
I20260811 10:15:28.766003 3784 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:15:28.823143 3767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:28.823367 3767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:28.823407 3767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:28.824606 3767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:28.824671 3767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:28.824714 3767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:15:28.826161 3767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:41307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=38691
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:28.826426 3767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:28.826653 3767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:28.832873 3811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:28.832983 3767 server_base.cc:1034] running on GCE node
W20260811 10:15:28.833117 3808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:28.832934 3814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:28.833428 3767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:28.833705 3767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:28.834896 3767 hybrid_clock.cc:648] HybridClock initialized: now 1786443328834890 us; error 34 us; skew 500 ppm
Aug 11 10:15:28 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:28.837919 3767 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:15:28.838374 3767 webserver.cc:460] Webserver started at http://127.8.238.1:38691/ using document root <none> and password file <none>
I20260811 10:15:28.838629 3767 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:28.838712 3767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:28.840194 3767 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:15:28.840921 3825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:28.841099 3767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:15:28.841208 3767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "ff9f7b37be68452bb83f552c26e68d61"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "01208b288a7c0c0f21885cf76159f65c"
server_key_iv: "f98d4746a822da73c0efb1102db3d6c1"
server_key_version: "encryptionkey@0"
I20260811 10:15:28.841593 3767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:28.856776 3767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:28.859104 3767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:28.859272 3767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:28.859515 3767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:28.860029 3836 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:15:28.861482 3767 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:15:28.861541 3767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:15:28.861572 3767 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:15:28.862223 3767 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:15:28.862269 3767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 10:15:28.862314 3836 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap starting.
I20260811 10:15:28.871301 3767 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:41307
I20260811 10:15:28.871868 3767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:15:28.872884 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 3767
I20260811 10:15:28.873287 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:34501
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=46353
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:15:28.880821 3836 log.cc:824] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:28.882977 4031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:41307 every 8 connection(s)
I20260811 10:15:28.885118 3836 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:15:28.885516 3836 tablet_bootstrap.cc:492] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap complete.
I20260811 10:15:28.886548 3836 ts_tablet_manager.cc:1397] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.015s
I20260811 10:15:28.887606 3836 raft_consensus.cc:348] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:28.887935 3836 raft_consensus.cc:729] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff9f7b37be68452bb83f552c26e68d61, State: Initialized, Role: FOLLOWER
I20260811 10:15:28.888146 3836 consensus_queue.cc:260] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:28.889554 3836 ts_tablet_manager.cc:1428] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
Aug 11 10:15:28 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:28.903699 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:28.872897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:51451 (local address 127.8.238.62:41953)
0811 10:15:28.879882 (+ 6985us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:28.879897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:28.881241 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 10:15:28.881644 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:28.881656 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:28.881747 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:28.881944 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:28.892861 (+ 10917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.893720 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.895365 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.896037 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.896602 (+ 565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:28.899245 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:28.899250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:28.899251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:28.899264 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:28.901574 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:28.902163 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:28.902167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:28.902167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:28.902322 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:28.902615 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:28.902616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:28.902617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:28.902930 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:28.903069 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:28.903372 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 10:15:28.903504 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6897,"thread_start_us":83,"threads_started":1}
I20260811 10:15:28.903699 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:28.872797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:51451)
0811 10:15:28.880024 (+ 7227us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:28.880094 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:28.881026 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:28.891881 (+ 10855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:28.891887 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:28.892348 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:28.892682 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:28.892691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.893851 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.893852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:28.895227 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:28.895231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.896153 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.896154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:28.896299 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:28.897055 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:28.897067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:28.899132 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:28.901685 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:28.901693 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:28.901696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:28.902049 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:28.902381 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:28.902382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:28.902383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:28.902534 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:28.903060 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:28.903067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:28.903201 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 10:15:28.903315 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 10:15:28.903504 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7169,"thread_start_us":100,"threads_started":1}
I20260811 10:15:28.904922 4033 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:28.905021 4033 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:28.905203 4033 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:28.905835 3687 ts_manager.cc:194] Registered new tserver with Master: ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307)
I20260811 10:15:28.906865 3687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:51451
I20260811 10:15:28.908198 4033 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
Aug 11 10:15:28 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.1@KRBTEST.COM
I20260811 10:15:28.916579 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:28.907309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41307 (local address 127.0.0.1:52052)
0811 10:15:28.907482 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:28.907523 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:28.907589 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:28.907817 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:28.907824 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:28.908181 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:28.908241 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:28.908244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.909468 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.909469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:28.910415 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:28.910418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.911026 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.911027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:28.911110 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:28.911626 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:28.911632 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:28.913225 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:28.915322 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:28.915328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:28.915330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:28.915576 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:28.915821 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:28.915822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:28.915823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:28.915893 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:28.916210 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:28.916217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:28.916267 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 10:15:28.916349 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 10:15:28.916481 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260811 10:15:28.916883 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:28.907387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.8.238.1:41307)
0811 10:15:28.907649 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:28.907653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:28.907664 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:15:28.907695 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:28.907697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:28.907712 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:28.907811 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:28.908341 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.909372 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.910515 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:28.910947 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:28.911022 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:28.913327 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:28.913335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:28.913335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:28.913342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:28.915227 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:28.915666 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:28.915669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:28.915670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:28.915729 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:28.915959 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:28.915960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:28.915960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:28.916110 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:28.916183 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:28.916701 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 10:15:28.916783 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":69,"threads_started":1}
I20260811 10:15:28.918578 3908 tablet_service.cc:1514] Processing DeleteTablet for tablet ed66e292ca504547972ee83b4e9ed901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:08 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:52052
I20260811 10:15:28.919660 4051 tablet_replica.cc:330] stopping tablet replica
I20260811 10:15:28.919838 4051 raft_consensus.cc:2229] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:15:28.919932 4051 raft_consensus.cc:2258] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:15:28.920275 4051 ts_tablet_manager.cc:1905] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:15:28.923033 4051 ts_tablet_manager.cc:1918] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:15:28.923118 4051 log.cc:1196] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal/wals/ed66e292ca504547972ee83b4e9ed901
I20260811 10:15:28.923358 4051 ts_tablet_manager.cc:1939] T ed66e292ca504547972ee83b4e9ed901 P ff9f7b37be68452bb83f552c26e68d61: Deleting consensus metadata
I20260811 10:15:28.923925 3660 catalog_manager.cc:4841] TS ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307): tablet ed66e292ca504547972ee83b4e9ed901 (table test-table [id=62285a9683f641d2a012a7092219a09c]) successfully deleted
W20260811 10:15:28.985021 4042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:28.985257 4042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:28.985297 4042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:28.986438 4042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:28.986505 4042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:28.986546 4042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:15:28.988029 4042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:34501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=46353
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:28.988306 4042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:28.988579 4042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:28.993662 4064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:28.993664 4065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:28.993654 4073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:28.993927 4042 server_base.cc:1034] running on GCE node
I20260811 10:15:28.994145 4042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:28.994400 4042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:28.995566 4042 hybrid_clock.cc:648] HybridClock initialized: now 1786443328995545 us; error 34 us; skew 500 ppm
Aug 11 10:15:28 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:28.998066 4042 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:15:28.998523 4042 webserver.cc:460] Webserver started at http://127.8.238.2:46353/ using document root <none> and password file <none>
I20260811 10:15:28.998770 4042 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:28.998852 4042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:29.000427 4042 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:15:29.001264 4085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.001456 4042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:29.001562 4042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "4424687e685b42908b51fb7bb0b4a4b2"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "2b70794c8e14f771a30e5d74c322a583"
server_key_iv: "05773ba64a54866030b59d41fae00e90"
server_key_version: "encryptionkey@0"
I20260811 10:15:29.001930 4042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:29.021806 4042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:29.023624 4042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:29.023823 4042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:29.024078 4042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:29.024397 4042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:29.024454 4042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.024490 4042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:29.024518 4042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.030808 4042 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:34501
I20260811 10:15:29.030869 4261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:34501 every 8 connection(s)
I20260811 10:15:29.031195 4042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:15:29 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443328, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:29.039692 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 4042
I20260811 10:15:29.040086 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:43881
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=42801
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:15:29.040817 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.031957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:44003)
0811 10:15:29.032118 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.032161 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.032653 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.032889 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.032892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.033450 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:29.033637 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.033642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.034357 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.034358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.035150 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.035152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.035537 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.035539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.035635 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.036060 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:29.036068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:29.037467 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:29.039188 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:29.039196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:29.039198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:29.039424 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:29.039678 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:29.039678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:29.039679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:29.039852 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:29.040324 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:29.040332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:29.040459 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.040546 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.040664 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260811 10:15:29.040843 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.032006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:44003 (local address 127.8.238.62:41953)
0811 10:15:29.032055 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.032057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.032250 (+ 193us) server_negotiation.cc:407] Connection header received
0811 10:15:29.032751 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.032752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.032767 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.032838 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:29.033736 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.034262 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.035238 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.035457 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.035517 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.037562 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:29.037564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:29.037564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:29.037571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:29.039092 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.039525 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.039527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.039527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.039604 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.039937 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.039938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.039938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.040096 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:29.040177 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.040597 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.040704 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:15:29.041659 4262 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:29.041767 4262 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:29.041971 4262 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:29.042425 3687 ts_manager.cc:194] Registered new tserver with Master: 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501)
I20260811 10:15:29.042811 3687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:44003
W20260811 10:15:29.125164 4266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:29.125362 4266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:29.125384 4266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:29.126521 4266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:29.126560 4266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:29.126586 4266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:15:29.128010 4266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:43881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=42801
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:29.128194 4266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:29.128374 4266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:29.133289 4277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:29.133285 4278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:29.133517 4282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:29.133450 4266 server_base.cc:1034] running on GCE node
I20260811 10:15:29.133700 4266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:29.133944 4266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:29.135107 4266 hybrid_clock.cc:648] HybridClock initialized: now 1786443329135097 us; error 44 us; skew 500 ppm
Aug 11 10:15:29 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:29.137534 4266 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:15:29.137941 4266 webserver.cc:460] Webserver started at http://127.8.238.3:42801/ using document root <none> and password file <none>
I20260811 10:15:29.138171 4266 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:29.138258 4266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:29.139541 4266 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:15:29.140165 4293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.140339 4266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.140442 4266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "60c75a4d543daf6eb109a7b0f04d243a"
server_key_iv: "841e44a788f8f42809945a0cf8aafc7b"
server_key_version: "encryptionkey@0"
I20260811 10:15:29.140761 4266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:29.166582 4266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:29.168601 4266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:29.168762 4266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:29.168994 4266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:29.169313 4266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:29.169381 4266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.169425 4266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:29.169453 4266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:29.175413 4266 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43881
I20260811 10:15:29.175496 4497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43881 every 8 connection(s)
I20260811 10:15:29.175845 4266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:15:29.176925 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 4266
Aug 11 10:15:29 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443329, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:15:29.185003 4509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.177315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.8.238.62:41953)
0811 10:15:29.177448 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.177450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.177612 (+ 162us) server_negotiation.cc:407] Connection header received
0811 10:15:29.177660 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.177660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.177670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.177731 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:29.178563 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.179301 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.180323 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.180514 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.180568 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.182064 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:29.182065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:29.182065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:29.182080 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:29.183704 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.184131 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.184132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.184133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.184201 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.184416 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.184417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.184417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.184564 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:29.184749 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.184800 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.184882 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260811 10:15:29.186661 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.176570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:44767 (local address 127.8.238.62:41953)
0811 10:15:29.176620 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.176622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.176914 (+ 292us) server_negotiation.cc:407] Connection header received
0811 10:15:29.177643 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.177645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.177658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.177725 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:29.179005 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.179578 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.180942 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.181140 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.181245 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.183699 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:29.183701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:29.183701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:29.183708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:29.185338 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.185861 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.185861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.185861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.185913 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.186185 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.186186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.186187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.186291 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:29.186349 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.186543 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.186603 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:15:29.186918 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.176536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:44767)
0811 10:15:29.176767 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.176820 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.177535 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.177794 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.177798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.178600 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:29.178918 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.178923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.179664 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.179665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.180858 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.180861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.181259 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.181260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.181401 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.181918 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:29.181927 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:29.183611 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:29.185405 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:29.185412 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:29.185414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:29.185731 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:29.185975 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:29.185975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:29.185976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:29.186113 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:29.186350 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:29.186353 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:29.186457 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.186533 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.186647 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260811 10:15:29.187387 4498 heartbeater.cc:347] Connected to a master server at 127.8.238.62:41953
I20260811 10:15:29.187469 4498 heartbeater.cc:464] Registering TS with master...
I20260811 10:15:29.187664 4498 heartbeater.cc:511] Master 127.8.238.62:41953 requested a full tablet report, sending...
I20260811 10:15:29.188045 3687 ts_manager.cc:194] Registered new tserver with Master: 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
I20260811 10:15:29.188413 3687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:44767
I20260811 10:15:29.197068 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:15:29.199669 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.197450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.8.238.62:41953)
0811 10:15:29.197508 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.197510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.197595 (+ 85us) server_negotiation.cc:407] Connection header received
0811 10:15:29.197647 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.197647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.197686 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.197748 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:15:29.197873 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.198329 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.199082 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.199255 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.199305 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.199455 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:15:29.199500 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 10:15:29.199619 (+ 119us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45956: token expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 10:15:29.199745 4041 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45956: token expired
W20260811 10:15:29.199910 3562 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.238.62:41953)
I20260811 10:15:29.205581 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.200197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.8.238.62:41953)
0811 10:15:29.200229 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.200231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.200239 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:15:29.200284 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.200285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.200292 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.200328 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:29.200784 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.201254 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.201903 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.202069 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.202118 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.203176 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:29.203177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:29.203177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:29.203183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:29.204671 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.204994 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.204995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.204995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.205049 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:29.205217 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:29.205217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:29.205217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:29.205309 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:29.205386 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.205472 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.205527 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260811 10:15:29.206875 3687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:15:29.208635 3687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:15:29.210908 3908 tablet_service.cc:1467] Processing CreateTablet for tablet 61528c0515a740de9a39c634ab988d13 (DEFAULT_TABLE table=test-table [id=df6e8b93d9a44016aa4a4e90bc463b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:29.211138 3908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61528c0515a740de9a39c634ab988d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:29.212744 3836 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap starting.
I20260811 10:15:29.213107 3836 tablet_bootstrap.cc:654] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:29.213689 3836 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: No bootstrap required, opened a new log
I20260811 10:15:29.213744 3836 ts_tablet_manager.cc:1397] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:29.214141 3836 raft_consensus.cc:348] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.214259 3836 raft_consensus.cc:374] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:29.214308 3836 raft_consensus.cc:729] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff9f7b37be68452bb83f552c26e68d61, State: Initialized, Role: FOLLOWER
I20260811 10:15:29.214357 3836 consensus_queue.cc:260] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.214458 3836 ts_tablet_manager.cc:1428] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 10:15:29.215262 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.210658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43881 (local address 127.0.0.1:47392)
0811 10:15:29.210762 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.210772 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.210813 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.211267 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.211267 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.211269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:29.211377 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.211382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.212840 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.212841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.214118 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.214122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.215053 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.215054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.215112 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.215129 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.215175 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.215209 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":90,"thread_start_us":67,"threads_started":1}
I20260811 10:15:29.215337 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.210880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47392 (local address 127.8.238.3:43881)
0811 10:15:29.211065 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.211069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.211078 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:15:29.211122 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.211124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.211141 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.211337 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:29.211450 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.212717 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.214217 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.214960 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.215059 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.215127 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.215209 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.215251 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260811 10:15:29.216022 4377 tablet_service.cc:1467] Processing CreateTablet for tablet 61528c0515a740de9a39c634ab988d13 (DEFAULT_TABLE table=test-table [id=df6e8b93d9a44016aa4a4e90bc463b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:29.216250 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.210576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:34501 (local address 127.0.0.1:50534)
0811 10:15:29.210610 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.210622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.210664 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.211264 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.211265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.211267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:29.211350 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.211353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.212832 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.212833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.214118 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.214122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.215992 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.215993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.216073 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.216077 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.216142 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.216187 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":59}
I20260811 10:15:29.216372 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.210751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.8.238.2:34501)
0811 10:15:29.210955 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.210960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.210970 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:15:29.211124 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.211126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.211141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.211499 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:29.211597 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.212734 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.215178 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.215904 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.215965 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.216045 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.216158 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.216268 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260811 10:15:29.216403 4377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61528c0515a740de9a39c634ab988d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:29.216960 4158 tablet_service.cc:1467] Processing CreateTablet for tablet 61528c0515a740de9a39c634ab988d13 (DEFAULT_TABLE table=test-table [id=df6e8b93d9a44016aa4a4e90bc463b00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:15:29.217187 4158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61528c0515a740de9a39c634ab988d13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:15:29.219357 4520 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Bootstrap starting.
I20260811 10:15:29.220012 4520 tablet_bootstrap.cc:654] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:29.220032 4521 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Bootstrap starting.
I20260811 10:15:29.220543 4521 tablet_bootstrap.cc:654] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Neither blocks nor log segments found. Creating new log.
I20260811 10:15:29.220593 4520 log.cc:824] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:29.221206 4521 log.cc:824] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:29.221463 4520 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: No bootstrap required, opened a new log
I20260811 10:15:29.221545 4520 ts_tablet_manager.cc:1397] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:15:29.221994 4521 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: No bootstrap required, opened a new log
I20260811 10:15:29.222078 4521 ts_tablet_manager.cc:1397] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:15:29.223428 4520 raft_consensus.cc:348] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.223567 4520 raft_consensus.cc:374] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:29.223592 4520 raft_consensus.cc:729] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b9a1fa12ad499fa46b3b652322dd5c, State: Initialized, Role: FOLLOWER
I20260811 10:15:29.223485 4521 raft_consensus.cc:348] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.223693 4521 raft_consensus.cc:374] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:15:29.223763 4521 raft_consensus.cc:729] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4424687e685b42908b51fb7bb0b4a4b2, State: Initialized, Role: FOLLOWER
I20260811 10:15:29.223740 4520 consensus_queue.cc:260] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.223889 4521 consensus_queue.cc:260] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.224109 4520 ts_tablet_manager.cc:1428] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 10:15:29.224133 4498 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
I20260811 10:15:29.224197 4521 ts_tablet_manager.cc:1428] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 10:15:29.224419 4262 heartbeater.cc:503] Master 127.8.238.62:41953 was elected leader, sending a full tablet report...
I20260811 10:15:29.260792 4045 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:29.260979 4045 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.261341 4045 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
I20260811 10:15:29.264513 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.261552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:34501 (local address 127.8.238.1:58677)
0811 10:15:29.261622 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.261637 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.261704 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.261930 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.261931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.261934 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:29.262021 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.262023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.262695 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.262696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.263552 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.263555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.264196 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.264198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.264284 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.264310 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.264375 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.264406 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":39}
I20260811 10:15:29.264516 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.261670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:58677 (local address 127.8.238.2:34501)
0811 10:15:29.261783 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.261785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.261796 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:15:29.261829 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.261830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.261843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.261912 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:29.262107 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.262608 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.263653 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.264110 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.264166 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.264175 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.264394 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.264442 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 10:15:29.264570 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.261674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:53145 (local address 127.8.238.3:43881)
0811 10:15:29.261740 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.261742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.261860 (+ 118us) server_negotiation.cc:407] Connection header received
0811 10:15:29.261921 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.261922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.261936 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.261998 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:15:29.262179 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.262628 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.263648 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.264190 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.264269 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.264279 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.264454 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.264494 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:15:29.264598 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.261552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43881 (local address 127.8.238.1:53145)
0811 10:15:29.261774 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.261785 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:29.261849 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:29.262008 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:29.262009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:29.262010 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:15:29.262098 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.262100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.262733 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.262733 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.263555 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:29.263557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.264290 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.264291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:29.264343 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.264346 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:15:29.264432 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 10:15:29.264467 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":79,"thread_start_us":109,"threads_started":1}
I20260811 10:15:29.264920 4187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61528c0515a740de9a39c634ab988d13" candidate_uuid: "ff9f7b37be68452bb83f552c26e68d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4424687e685b42908b51fb7bb0b4a4b2" is_pre_election: true
I20260811 10:15:29.264918 4415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61528c0515a740de9a39c634ab988d13" candidate_uuid: "ff9f7b37be68452bb83f552c26e68d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" is_pre_election: true
I20260811 10:15:29.265080 4415 raft_consensus.cc:2454] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff9f7b37be68452bb83f552c26e68d61 in term 0.
I20260811 10:15:29.265080 4187 raft_consensus.cc:2454] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff9f7b37be68452bb83f552c26e68d61 in term 0.
I20260811 10:15:29.265311 3827 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b9a1fa12ad499fa46b3b652322dd5c, ff9f7b37be68452bb83f552c26e68d61; no voters:
I20260811 10:15:29.265466 4045 raft_consensus.cc:2781] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:15:29.265534 4045 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:15:29.265552 4045 raft_consensus.cc:3037] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:29.266335 4045 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.266491 4045 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 1 election: Requested vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
I20260811 10:15:29.266624 4187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61528c0515a740de9a39c634ab988d13" candidate_uuid: "ff9f7b37be68452bb83f552c26e68d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4424687e685b42908b51fb7bb0b4a4b2"
I20260811 10:15:29.266636 4415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61528c0515a740de9a39c634ab988d13" candidate_uuid: "ff9f7b37be68452bb83f552c26e68d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
I20260811 10:15:29.266709 4415 raft_consensus.cc:3037] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:29.266709 4187 raft_consensus.cc:3037] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:15:29.267616 4415 raft_consensus.cc:2454] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff9f7b37be68452bb83f552c26e68d61 in term 1.
I20260811 10:15:29.267616 4187 raft_consensus.cc:2454] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff9f7b37be68452bb83f552c26e68d61 in term 1.
I20260811 10:15:29.267841 3827 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b9a1fa12ad499fa46b3b652322dd5c, ff9f7b37be68452bb83f552c26e68d61; no voters:
I20260811 10:15:29.267995 4045 raft_consensus.cc:2781] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:15:29.268157 4045 raft_consensus.cc:686] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 LEADER]: Becoming Leader. State: Replica: ff9f7b37be68452bb83f552c26e68d61, State: Running, Role: LEADER
I20260811 10:15:29.268251 4045 consensus_queue.cc:237] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:29.269065 3685 catalog_manager.cc:5495] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "ff9f7b37be68452bb83f552c26e68d61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } health_report { overall_health: HEALTHY } } }
W20260811 10:15:29.281757 4264 tablet.cc:2369] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:15:29.287902 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.285576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52066 (local address 127.8.238.1:41307)
0811 10:15:29.285638 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:29.285640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:29.285671 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:15:29.285707 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:29.285708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:29.285726 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:29.285779 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:15:29.285900 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.286382 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.287102 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:29.287313 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:29.287377 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:29.287548 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:15:29.287649 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:15:29.287705 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:29.287824 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 10:15:29.287846 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:15:29.289520 4415 raft_consensus.cc:1261] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Refusing update from remote peer ff9f7b37be68452bb83f552c26e68d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:15:29.289520 4187 raft_consensus.cc:1261] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Refusing update from remote peer ff9f7b37be68452bb83f552c26e68d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:15:29.289904 4045 consensus_queue.cc:1035] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:15:29.290093 4527 consensus_queue.cc:1035] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:15:29.292526 4533 mvcc.cc:205] Tried to move back new op lower bound from 7317271876766969856 to 7317271876683411456. Current Snapshot: MvccSnapshot[applied={T|T < 7317271876766969856}]
I20260811 10:15:29.294574 3687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:15:29.294672 3687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:15:29.295436 3687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Sending DeleteTablet for 3 replicas of tablet 61528c0515a740de9a39c634ab988d13
I20260811 10:15:29.295809 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 3767
I20260811 10:15:29.295845 4377 tablet_service.cc:1514] Processing DeleteTablet for tablet 61528c0515a740de9a39c634ab988d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:29 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:47392
I20260811 10:15:29.295859 4158 tablet_service.cc:1514] Processing DeleteTablet for tablet 61528c0515a740de9a39c634ab988d13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:15:29 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:50534
I20260811 10:15:29.298792 4550 tablet_replica.cc:330] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: stopping tablet replica
I20260811 10:15:29.298934 4550 raft_consensus.cc:2229] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:15:29.299086 4550 raft_consensus.cc:2258] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:15:29.299295 4549 tablet_replica.cc:330] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: stopping tablet replica
I20260811 10:15:29.299433 4549 raft_consensus.cc:2229] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:15:29.299574 4549 raft_consensus.cc:2258] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:15:29.300662 4550 ts_tablet_manager.cc:1905] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:15:29.303166 4550 ts_tablet_manager.cc:1918] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:15:29.303277 4550 log.cc:1196] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/61528c0515a740de9a39c634ab988d13
I20260811 10:15:29.303690 4550 ts_tablet_manager.cc:1939] T 61528c0515a740de9a39c634ab988d13 P 4424687e685b42908b51fb7bb0b4a4b2: Deleting consensus metadata
I20260811 10:15:29.304184 4549 ts_tablet_manager.cc:1905] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:15:29.305308 3661 catalog_manager.cc:4841] TS 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): tablet 61528c0515a740de9a39c634ab988d13 (table test-table [id=df6e8b93d9a44016aa4a4e90bc463b00]) successfully deleted
I20260811 10:15:29.307231 4549 ts_tablet_manager.cc:1918] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:15:29.307309 4549 log.cc:1196] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/61528c0515a740de9a39c634ab988d13
I20260811 10:15:29.307557 4549 ts_tablet_manager.cc:1939] T 61528c0515a740de9a39c634ab988d13 P 07b9a1fa12ad499fa46b3b652322dd5c: Deleting consensus metadata
I20260811 10:15:29.310607 3655 catalog_manager.cc:4841] TS 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): tablet 61528c0515a740de9a39c634ab988d13 (table test-table [id=df6e8b93d9a44016aa4a4e90bc463b00]) successfully deleted
W20260811 10:15:29.315299 3660 connection.cc:535] client connection to 127.8.238.1:41307 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 10:15:29.315460 3660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:15:29.315826 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 4042
I20260811 10:15:29.316001 4048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:29.315807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:41307 (local address 127.0.0.1:52078)
0811 10:15:29.315887 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:29.315960 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 10:15:29.316135 3660 catalog_manager.cc:4584] TS ff9f7b37be68452bb83f552c26e68d61 (127.8.238.1:41307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61528c0515a740de9a39c634ab988d13: Network error: Client connection negotiation failed: client connection to 127.8.238.1:41307: connect: Connection refused (error 111)
I20260811 10:15:29.323951 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 4266
I20260811 10:15:29.333850 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 3627
I20260811 10:15:49.343055 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--webserver_interface=127.8.238.62
--webserver_port=42757
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:41953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:49.427336 4552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:49.427583 4552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:49.427621 4552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:49.428797 4552 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 10:15:49.428877 4552 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 10:15:49.428902 4552 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 10:15:49.428929 4552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:15:49.428961 4552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:49.428987 4552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:49.429018 4552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:15:49.429049 4552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:15:49.430528 4552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:41953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:41953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=42757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:49.430809 4552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:49.431046 4552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:49.436460 4565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.436486 4558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:49.436519 4552 server_base.cc:1034] running on GCE node
W20260811 10:15:49.436486 4559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:49.436931 4552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:49.437250 4552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:49.438405 4552 hybrid_clock.cc:648] HybridClock initialized: now 1786443349438382 us; error 40 us; skew 500 ppm
Aug 11 10:15:49 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:49.441149 4552 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:15:49.441550 4552 webserver.cc:460] Webserver started at http://127.8.238.62:42757/ using document root <none> and password file <none>
I20260811 10:15:49.441756 4552 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:49.441797 4552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:49.443071 4552 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:49.443724 4580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.443972 4552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:49.444073 4552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "db3faf0c95b5411e919736cc1ef80a31"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "44ed8aa508ef2755a2b3dd668500a972"
server_key_iv: "4ce51635781407bb25295642103fe491"
server_key_version: "encryptionkey@0"
I20260811 10:15:49.444425 4552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:49.468874 4552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:49.471035 4552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:49.471194 4552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:49.475418 4552 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:41953
I20260811 10:15:49.475453 4667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:41953 every 8 connection(s)
I20260811 10:15:49.476056 4552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:15:49.477417 4668 sys_catalog.cc:263] Verifying existing consensus state
I20260811 10:15:49.478102 4668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap starting.
I20260811 10:15:49.479359 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 4552
I20260811 10:15:49.479696 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:41307
--local_ip_for_outbound_sockets=127.8.238.1
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=38691
--webserver_interface=127.8.238.1
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:15:49.489187 4668 log.cc:824] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:49.491505 4668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:15:49.491897 4668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Bootstrap complete.
I20260811 10:15:49.494259 4668 raft_consensus.cc:348] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:49.494517 4668 raft_consensus.cc:729] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Initialized, Role: FOLLOWER
I20260811 10:15:49.494727 4668 consensus_queue.cc:260] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:49.494814 4668 raft_consensus.cc:388] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:15:49.494869 4668 raft_consensus.cc:482] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:15:49.494920 4668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 2 FOLLOWER]: Advancing to term 3
I20260811 10:15:49.495980 4668 raft_consensus.cc:504] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:49.496101 4668 leader_election.cc:302] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db3faf0c95b5411e919736cc1ef80a31; no voters:
I20260811 10:15:49.496299 4668 leader_election.cc:288] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 10:15:49.496340 4675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 10:15:49.496461 4675 raft_consensus.cc:686] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [term 3 LEADER]: Becoming Leader. State: Replica: db3faf0c95b5411e919736cc1ef80a31, State: Running, Role: LEADER
I20260811 10:15:49.496552 4675 consensus_queue.cc:237] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } }
I20260811 10:15:49.496590 4668 sys_catalog.cc:564] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:15:49.496825 4675 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db3faf0c95b5411e919736cc1ef80a31. Latest consensus state: current_term: 3 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:49.496918 4675 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:49.496831 4676 sys_catalog.cc:455] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "db3faf0c95b5411e919736cc1ef80a31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db3faf0c95b5411e919736cc1ef80a31" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 41953 } } }
I20260811 10:15:49.497110 4676 sys_catalog.cc:458] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31 [sys.catalog]: This master's current role is: LEADER
I20260811 10:15:49.497130 4680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:15:49.498035 4680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:15:49.498380 4680 catalog_manager.cc:1259] Loaded cluster ID: 764bdeb443234f0584d01a1ac53297a0
I20260811 10:15:49.498456 4680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:15:49.499771 4680 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:15:49.499959 4680 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Loaded TSK: 0
I20260811 10:15:49.500005 4680 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db3faf0c95b5411e919736cc1ef80a31: Loaded TSK: 1
F20260811 10:15:49.500229 4680 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3bf68439ad google::LogMessage::Fail() at ??:0
@ 0x7f3bf684688a google::LogMessage::SendToLog() at ??:0
@ 0x7f3bf68434de google::LogMessage::Flush() at ??:0
@ 0x7f3bf6844a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3bf8fb0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3bf8fadd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3bfe7a11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3bfe7a148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3bfe7a1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3bfe767c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3bfe771ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3bf87110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3bf8706954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3bf956a6db start_thread at ??:0
@ 0x7f3bf534161f clone at ??:0
W20260811 10:15:49.580793 4673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:49.581034 4673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:49.581074 4673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:49.582216 4673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:49.582283 4673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:49.582329 4673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:15:49.584044 4673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:41307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=38691
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:49.584291 4673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:49.584549 4673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:49.590601 4738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.590624 4735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.590628 4736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:49.591289 4673 server_base.cc:1034] running on GCE node
I20260811 10:15:49.591517 4673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:49.591811 4673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:49.593007 4673 hybrid_clock.cc:648] HybridClock initialized: now 1786443349592981 us; error 38 us; skew 500 ppm
Aug 11 10:15:49 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:49.596180 4673 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:15:49.596668 4673 webserver.cc:460] Webserver started at http://127.8.238.1:38691/ using document root <none> and password file <none>
I20260811 10:15:49.596925 4673 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:49.596972 4673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:49.598328 4673 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:15:49.600409 4750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.600612 4673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:15:49.600718 4673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "ff9f7b37be68452bb83f552c26e68d61"
format_stamp: "Formatted at 2026-08-11 10:15:07 on dist-test-slave-7f01"
server_key: "01208b288a7c0c0f21885cf76159f65c"
server_key_iv: "f98d4746a822da73c0efb1102db3d6c1"
server_key_version: "encryptionkey@0"
I20260811 10:15:49.601092 4673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:49.613705 4673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:49.616149 4673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:49.616329 4673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:49.616607 4673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:49.617057 4763 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 10:15:49.619192 4673 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 10:15:49.619262 4673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 10:15:49.619308 4673 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 10:15:49.620064 4673 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 10:15:49.620121 4673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 10:15:49.620189 4763 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap starting.
I20260811 10:15:49.628131 4673 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:41307
I20260811 10:15:49.628526 4673 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:15:49.629143 4946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:41307 every 8 connection(s)
I20260811 10:15:49.632707 4763 log.cc:824] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Log is configured to *not* fsync() on all Append() calls
I20260811 10:15:49.635593 4763 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 10:15:49.635991 4763 tablet_bootstrap.cc:492] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Bootstrap complete.
I20260811 10:15:49.636725 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 4673
I20260811 10:15:49.637037 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:34501
--local_ip_for_outbound_sockets=127.8.238.2
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=46353
--webserver_interface=127.8.238.2
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:15:49.637403 4763 ts_tablet_manager.cc:1397] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.013s
I20260811 10:15:49.638878 4763 raft_consensus.cc:348] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:49.639215 4763 raft_consensus.cc:729] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff9f7b37be68452bb83f552c26e68d61, State: Initialized, Role: FOLLOWER
I20260811 10:15:49.639396 4763 consensus_queue.cc:260] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:49.639688 4763 ts_tablet_manager.cc:1428] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
W20260811 10:15:49.713680 4952 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:15:49.629314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:53785)
0811 10:15:49.629522 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.629577 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:49.630395 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:49.713442 (+ 83047us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
W20260811 10:15:49.713986 4948 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:41953 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:41953: Client connection negotiation failed: client connection to 127.8.238.62:41953: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 10:15:49.714372 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.714240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:49537)
0811 10:15:49.714299 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.714341 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 10:15:49.714800 4952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.714712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:40603)
0811 10:15:49.714758 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.714783 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:15:49.714931 4948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 10:15:49.723534 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:49.723724 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:49.723816 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:49.724992 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:49.725035 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:49.725061 4955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:15:49.726503 4955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:34501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=46353
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:49.726773 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:49.727007 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:49.732053 4979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.732065 4976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.732069 4973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:49.732275 4955 server_base.cc:1034] running on GCE node
I20260811 10:15:49.732440 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:49.732621 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:49.733758 4955 hybrid_clock.cc:648] HybridClock initialized: now 1786443349733732 us; error 40 us; skew 500 ppm
Aug 11 10:15:49 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:49.736214 4955 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:15:49.736654 4955 webserver.cc:460] Webserver started at http://127.8.238.2:46353/ using document root <none> and password file <none>
I20260811 10:15:49.736884 4955 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:49.736956 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:49.738261 4955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:15:49.738910 4995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.739104 4955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:15:49.739209 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "4424687e685b42908b51fb7bb0b4a4b2"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "2b70794c8e14f771a30e5d74c322a583"
server_key_iv: "05773ba64a54866030b59d41fae00e90"
server_key_version: "encryptionkey@0"
I20260811 10:15:49.739548 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:49.746598 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:49.748448 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:49.748606 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:49.748823 4955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:49.749130 4955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:49.749186 4955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.749226 4955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:49.749254 4955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.755515 4955 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:34501
I20260811 10:15:49.755573 5198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:34501 every 8 connection(s)
I20260811 10:15:49.755973 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:15:49.756937 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.756582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:42591)
0811 10:15:49.756737 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.756802 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
W20260811 10:15:49.757321 5199 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:41953 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:41953: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
I20260811 10:15:49.757613 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.757505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:45529)
0811 10:15:49.757557 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.757583 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:15:49.757892 5207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.757803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:59027)
0811 10:15:49.757848 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.757871 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 10:15:49.758008 5199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:15:49.759711 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 4955
I20260811 10:15:49.760027 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:43881
--local_ip_for_outbound_sockets=127.8.238.3
--tserver_master_addrs=127.8.238.62:41953
--webserver_port=42801
--webserver_interface=127.8.238.3
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:43629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:15:49.845472 5208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:15:49.845674 5208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:15:49.845696 5208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:15:49.846776 5208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:15:49.846812 5208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:15:49.846836 5208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:15:49.848273 5208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:43629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:43881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=42801
--tserver_master_addrs=127.8.238.62:41953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:15:49.848464 5208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:15:49.848636 5208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:15:49.853683 5217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.853700 5223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:15:49.853721 5221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:15:49.854023 5208 server_base.cc:1034] running on GCE node
I20260811 10:15:49.854216 5208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:15:49.854449 5208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:15:49.855592 5208 hybrid_clock.cc:648] HybridClock initialized: now 1786443349855580 us; error 26 us; skew 500 ppm
Aug 11 10:15:49 dist-test-slave-7f01 krb5kdc[2656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:15:49.858098 5208 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:15:49.858528 5208 webserver.cc:460] Webserver started at http://127.8.238.3:42801/ using document root <none> and password file <none>
I20260811 10:15:49.858750 5208 fs_manager.cc:359] Metadata directory not provided
I20260811 10:15:49.858819 5208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:15:49.860203 5208 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:15:49.860795 5234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.860985 5208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.861093 5208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "07b9a1fa12ad499fa46b3b652322dd5c"
format_stamp: "Formatted at 2026-08-11 10:15:08 on dist-test-slave-7f01"
server_key: "60c75a4d543daf6eb109a7b0f04d243a"
server_key_iv: "841e44a788f8f42809945a0cf8aafc7b"
server_key_version: "encryptionkey@0"
I20260811 10:15:49.861512 5208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:15:49.870651 5208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:15:49.872804 5208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:15:49.872985 5208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:15:49.873245 5208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:15:49.873581 5208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:15:49.873646 5208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.873689 5208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:15:49.873720 5208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:15:49.882437 5208 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:43881
I20260811 10:15:49.882699 5435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:43881 every 8 connection(s)
I20260811 10:15:49.882942 5208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:15:49.884298 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.883709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:52825)
0811 10:15:49.883919 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.884057 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
W20260811 10:15:49.885186 5441 heartbeater.cc:644] Failed to heartbeat to 127.8.238.62:41953 (0 consecutive failures): Network error: Failed to ping master at 127.8.238.62:41953: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
I20260811 10:15:49.885892 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 5208
I20260811 10:15:49.886427 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.886279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:33063)
0811 10:15:49.886327 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.886393 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 10:15:49.886947 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:49.886842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:32837)
0811 10:15:49.886896 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:49.886922 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 10:15:49.887115 5441 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 10:15:50.715571 5450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:50.715317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:56263)
0811 10:15:50.715461 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:50.715499 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260811 10:15:50.758502 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:50.758295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:49983)
0811 10:15:50.758420 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:50.758454 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260811 10:15:50.887622 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:50.887404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:35717)
0811 10:15:50.887525 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:50.887559 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260811 10:15:51.388789 5456 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:51.388970 5456 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:51.389328 5456 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
Aug 11 10:15:51 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.3@KRBTEST.COM
Aug 11 10:15:51 dist-test-slave-7f01 krb5kdc[2656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443349, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.2@KRBTEST.COM
I20260811 10:15:51.400337 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.389615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:37699 (local address 127.8.238.3:43881)
0811 10:15:51.389809 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:51.389823 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:51.389850 (+ 27us) server_negotiation.cc:407] Connection header received
0811 10:15:51.390352 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:51.390355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:51.390397 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:51.390564 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:51.391402 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.392498 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.393536 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.393960 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.394082 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:51.396578 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:51.396582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:51.396582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:51.396592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:51.398488 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399080 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:51.399084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:51.399084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:51.399191 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399448 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:51.399449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:51.399449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:51.399667 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:51.399801 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:51.400088 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 10:15:51.400218 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":69,"threads_started":1}
I20260811 10:15:51.400326 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.389503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:34501 (local address 127.8.238.1:35713)
0811 10:15:51.389638 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:51.389647 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:51.389696 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:51.390666 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:51.390667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:51.391022 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:51.391261 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:51.391268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.392649 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.392650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:51.393434 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:51.393436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.394052 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.394054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:51.394173 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:51.394839 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:51.394846 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:51.396855 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:51.398725 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:51.398726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:51.398726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:51.398972 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:51.399213 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:51.399214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:51.399346 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:51.399760 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:51.399767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:51.399905 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 10:15:51.400014 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 10:15:51.400213 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":112,"thread_start_us":75,"threads_started":1}
I20260811 10:15:51.400381 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.389727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:35713 (local address 127.8.238.2:34501)
0811 10:15:51.389987 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 10:15:51.389997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:15:51.390025 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:15:51.390461 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:15:51.390464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:15:51.390500 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:15:51.390682 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:15:51.391390 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.392551 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.393541 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.393953 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.394088 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:51.396978 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:15:51.396982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:15:51.396982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:15:51.396992 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:15:51.398637 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399037 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:51.399040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:51.399041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:51.399144 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399418 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:15:51.399419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:15:51.399420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:15:51.399663 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:15:51.399780 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:15:51.400091 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 10:15:51.400239 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":85,"threads_started":1}
I20260811 10:15:51.400444 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.389503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:43881 (local address 127.8.238.1:37699)
0811 10:15:51.389620 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:51.389632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:15:51.389696 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:15:51.390609 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:15:51.390613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:15:51.391021 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:15:51.391261 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:51.391267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.392601 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.392602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:51.393440 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:15:51.393443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:15:51.394058 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:15:51.394059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:15:51.394179 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:15:51.394791 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:15:51.394799 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:15:51.396471 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:15:51.398579 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:51.398588 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:51.398590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:51.398981 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:51.399280 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:15:51.399281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:15:51.399281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:15:51.399352 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:15:51.399783 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:15:51.399784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:15:51.399903 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 10:15:51.400016 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 10:15:51.400308 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":184,"thread_start_us":52,"threads_started":1}
W20260811 10:15:51.401855 4752 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
W20260811 10:15:51.401957 4758 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
I20260811 10:15:51.402026 4758 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9f7b37be68452bb83f552c26e68d61; no voters: 07b9a1fa12ad499fa46b3b652322dd5c, 4424687e685b42908b51fb7bb0b4a4b2
I20260811 10:15:51.402163 5456 raft_consensus.cc:2726] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:15:51.716398 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.716218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:58443)
0811 10:15:51.716300 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:51.716348 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 10:15:51.759239 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.758970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:38587)
0811 10:15:51.759133 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:51.759187 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260811 10:15:51.888335 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:51.888078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:33891)
0811 10:15:51.888229 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:51.888280 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260811 10:15:52.717157 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:52.716926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:37499)
0811 10:15:52.717058 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:52.717096 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260811 10:15:52.759970 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:52.759705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:34265)
0811 10:15:52.759889 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:52.759927 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260811 10:15:52.889022 5470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:52.888780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:32917)
0811 10:15:52.888931 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:52.888972 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260811 10:15:53.373407 5471 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:53.373533 5471 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:53.373713 5471 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
W20260811 10:15:53.374258 4752 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
W20260811 10:15:53.374342 4758 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
I20260811 10:15:53.374408 4758 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9f7b37be68452bb83f552c26e68d61; no voters: 07b9a1fa12ad499fa46b3b652322dd5c, 4424687e685b42908b51fb7bb0b4a4b2
I20260811 10:15:53.374521 5471 raft_consensus.cc:2726] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:15:53.717900 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:53.717661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:53147)
0811 10:15:53.717803 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:53.717843 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260811 10:15:53.760676 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:53.760449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:38467)
0811 10:15:53.760577 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:53.760616 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260811 10:15:53.889797 5479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:53.889541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:57859)
0811 10:15:53.889696 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:53.889739 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260811 10:15:54.718730 5480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:54.718468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:36007)
0811 10:15:54.718630 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:54.718673 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260811 10:15:54.761418 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:54.761203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:57213)
0811 10:15:54.761328 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:54.761365 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260811 10:15:54.890522 5482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:54.890283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:44067)
0811 10:15:54.890425 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:54.890469 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260811 10:15:55.719556 5483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:55.719324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:45123)
0811 10:15:55.719461 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:55.719500 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260811 10:15:55.762125 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:55.761885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:50533)
0811 10:15:55.762037 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:55.762076 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
I20260811 10:15:55.891232 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:55.891001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:36547)
0811 10:15:55.891145 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:55.891181 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260811 10:15:56.124871 5490 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:56.124995 5490 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:56.125214 5490 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
W20260811 10:15:56.125653 4752 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
W20260811 10:15:56.125779 4758 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
I20260811 10:15:56.125878 4758 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9f7b37be68452bb83f552c26e68d61; no voters: 07b9a1fa12ad499fa46b3b652322dd5c, 4424687e685b42908b51fb7bb0b4a4b2
I20260811 10:15:56.125969 5490 raft_consensus.cc:2726] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:15:56.720450 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:56.720156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:52969)
0811 10:15:56.720310 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:56.720373 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260811 10:15:56.762796 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:56.762569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:40003)
0811 10:15:56.762716 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:56.762752 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260811 10:15:56.892020 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:56.891719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:46043)
0811 10:15:56.891933 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:56.891972 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":146,"threads_started":1}
I20260811 10:15:57.697235 5502 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:15:57.697350 5502 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:15:57.697558 5502 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
W20260811 10:15:57.697974 4752 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
W20260811 10:15:57.698058 4758 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
I20260811 10:15:57.698148 4758 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9f7b37be68452bb83f552c26e68d61; no voters: 07b9a1fa12ad499fa46b3b652322dd5c, 4424687e685b42908b51fb7bb0b4a4b2
I20260811 10:15:57.698233 5502 raft_consensus.cc:2726] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:15:57.721210 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:57.720945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:54309)
0811 10:15:57.721118 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:57.721156 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":70,"threads_started":1}
I20260811 10:15:57.763523 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:57.763289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:47109)
0811 10:15:57.763444 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:57.763479 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260811 10:15:57.892803 5513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:57.892536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:56955)
0811 10:15:57.892702 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:57.892752 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260811 10:15:58.722029 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:58.721759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:51481)
0811 10:15:58.721918 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:58.721973 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260811 10:15:58.764361 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:58.764078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:54229)
0811 10:15:58.764231 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:58.764301 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":63,"threads_started":1}
I20260811 10:15:58.893523 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:58.893275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:59299)
0811 10:15:58.893433 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:58.893473 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260811 10:15:59.722781 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:59.722540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:51935)
0811 10:15:59.722682 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:59.722722 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260811 10:15:59.765041 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:59.764791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:40229)
0811 10:15:59.764948 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:59.764981 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260811 10:15:59.894261 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:15:59.894010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:55251)
0811 10:15:59.894171 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 10:15:59.894212 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260811 10:16:00.723584 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:00.723315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:44887)
0811 10:16:00.723453 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:00.723495 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":131,"threads_started":1}
I20260811 10:16:00.765725 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:00.765461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:52679)
0811 10:16:00.765645 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:00.765679 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260811 10:16:00.895002 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:00.894763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:44901)
0811 10:16:00.894912 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:00.894949 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260811 10:16:01.724493 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:01.724189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:59445)
0811 10:16:01.724370 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:01.724432 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260811 10:16:01.766515 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:01.766239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:48107)
0811 10:16:01.766412 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:01.766462 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260811 10:16:01.895870 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:01.895557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:41285)
0811 10:16:01.895695 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:01.895806 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260811 10:16:02.725354 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:02.725087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:41755)
0811 10:16:02.725250 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:02.725298 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260811 10:16:02.767323 5535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:02.767057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:40203)
0811 10:16:02.767226 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:02.767273 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260811 10:16:02.896733 5537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:02.896461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:49017)
0811 10:16:02.896616 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:02.896678 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260811 10:16:03.726091 5538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:03.725847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:46481)
0811 10:16:03.726001 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:03.726038 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260811 10:16:03.768042 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:03.767826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:34445)
0811 10:16:03.767957 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:03.767991 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260811 10:16:03.897480 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:03.897258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:55819)
0811 10:16:03.897394 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:03.897430 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 10:16:04.257042 5544 raft_consensus.cc:482] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:04.257161 5544 raft_consensus.cc:504] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4424687e685b42908b51fb7bb0b4a4b2" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 34501 } } peers { permanent_uuid: "07b9a1fa12ad499fa46b3b652322dd5c" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 43881 } } peers { permanent_uuid: "ff9f7b37be68452bb83f552c26e68d61" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 41307 } }
I20260811 10:16:04.257346 5544 leader_election.cc:288] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501), 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881)
W20260811 10:16:04.257774 4758 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4424687e685b42908b51fb7bb0b4a4b2 (127.8.238.2:34501): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
W20260811 10:16:04.257866 4752 leader_election.cc:341] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b9a1fa12ad499fa46b3b652322dd5c (127.8.238.3:43881): Not found: Tablet not found: 61528c0515a740de9a39c634ab988d13
I20260811 10:16:04.257921 4752 leader_election.cc:302] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff9f7b37be68452bb83f552c26e68d61; no voters: 07b9a1fa12ad499fa46b3b652322dd5c, 4424687e685b42908b51fb7bb0b4a4b2
I20260811 10:16:04.258008 5544 raft_consensus.cc:2726] T 61528c0515a740de9a39c634ab988d13 P ff9f7b37be68452bb83f552c26e68d61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 10:16:04.726887 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:04.726629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.1:39313)
0811 10:16:04.726778 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:04.726811 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260811 10:16:04.768707 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:04.768496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.2:43165)
0811 10:16:04.768622 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:04.768657 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260811 10:16:04.891959 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 4673
I20260811 10:16:04.900432 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:04.898679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:41953 (local address 127.8.238.3:33861)
0811 10:16:04.900345 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:04.900379 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.62:41953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1642,"thread_start_us":666,"threads_started":1}
I20260811 10:16:04.911116 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 4955
I20260811 10:16:04.921584 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 5208
2026-08-11T10:16:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58768 ms)
[----------] 30 tests from SecurityITest (240866 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:04 dist-test-slave-7f01 krb5kdc[5551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:04 dist-test-slave-7f01 krb5kdc[5551](info): set up 2 sockets
Aug 11 10:16:04 dist-test-slave-7f01 krb5kdc[5551](info): commencing operation
krb5kdc: starting...
W20260811 10:16:06.203706 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:06.330801 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:33579
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:06.413196 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:06.413448 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:06.413487 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:06.414651 5580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:06.414724 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:06.414748 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:06.414775 5580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:06.414808 5580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:06.416332 5580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:06.416599 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:06.416827 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:16:06.422015 5580 server_base.cc:1034] running on GCE node
W20260811 10:16:06.421928 5599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.421979 5595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.421973 5596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.422501 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:06.422835 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:06.424003 5580 hybrid_clock.cc:648] HybridClock initialized: now 1786443366423987 us; error 43 us; skew 500 ppm
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:06.426590 5580 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:06.427079 5580 webserver.cc:460] Webserver started at http://127.8.238.62:46373/ using document root <none> and password file <none>
I20260811 10:16:06.427309 5580 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:06.427383 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:06.427524 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:06.428623 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "25e20489dbaa461b9343e6b299104e60"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "65c744245d019515cdb6e9a6c1d3e476"
server_key_iv: "34fb50b061081b0b43b37b0ee352706e"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.429009 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "25e20489dbaa461b9343e6b299104e60"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "65c744245d019515cdb6e9a6c1d3e476"
server_key_iv: "34fb50b061081b0b43b37b0ee352706e"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.430374 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:06.431145 5610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.431350 5580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:06.431463 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "25e20489dbaa461b9343e6b299104e60"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "65c744245d019515cdb6e9a6c1d3e476"
server_key_iv: "34fb50b061081b0b43b37b0ee352706e"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.431555 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:06.443427 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:06.445338 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:06.445513 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:06.449753 5580 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:33579
I20260811 10:16:06.449790 5701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:33579 every 8 connection(s)
I20260811 10:16:06.450135 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:06.451011 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:06.453405 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: Bootstrap starting.
I20260811 10:16:06.454028 5704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:06.454332 5704 log.cc:824] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:06.455062 5704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: No bootstrap required, opened a new log
I20260811 10:16:06.455798 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 5580
I20260811 10:16:06.455919 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:06.456123 9144 external_mini_cluster.cc:1428] Setting key 4fed6e0e772bbf3fe79cc38cebf9ce5c
I20260811 10:16:06.457334 5704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } }
I20260811 10:16:06.457515 5704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:06.457557 5704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e20489dbaa461b9343e6b299104e60, State: Initialized, Role: FOLLOWER
I20260811 10:16:06.457701 5704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } }
I20260811 10:16:06.457796 5704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:06.457831 5704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:06.457886 5704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:06.458895 5704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } }
I20260811 10:16:06.458992 5704 leader_election.cc:302] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25e20489dbaa461b9343e6b299104e60; no voters:
I20260811 10:16:06.459182 5704 leader_election.cc:288] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:06.459234 5715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:06.459383 5715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [term 1 LEADER]: Becoming Leader. State: Replica: 25e20489dbaa461b9343e6b299104e60, State: Running, Role: LEADER
I20260811 10:16:06.459481 5704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:06.459534 5715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } }
I20260811 10:16:06.459982 5718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e20489dbaa461b9343e6b299104e60. Latest consensus state: current_term: 1 leader_uuid: "25e20489dbaa461b9343e6b299104e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } } }
I20260811 10:16:06.460073 5718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:06.460306 5727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:06.460919 5727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:06.460932 5719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25e20489dbaa461b9343e6b299104e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e20489dbaa461b9343e6b299104e60" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 33579 } } }
I20260811 10:16:06.461376 5719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:06.463040 5727 catalog_manager.cc:1347] Generated new cluster ID: 00fd025c52694d7d931ced350d733977
I20260811 10:16:06.463099 5727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:06.468153 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.456656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.8.238.62:33579)
0811 10:16:06.456835 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:06.456844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:06.456894 (+ 50us) server_negotiation.cc:407] Connection header received
0811 10:16:06.457316 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:06.457322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:06.457369 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:06.457478 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:06.458336 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.459490 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.460672 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.461267 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.461388 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.464343 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:06.464347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:06.464347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:06.464358 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:06.466465 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.466844 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.466847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.466847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.467002 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.467208 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.467209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.467209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.467523 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:06.467680 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:06.467780 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 10:16:06.467940 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260811 10:16:06.487787 5727 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:06.488534 5727 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:06.492159 5727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: Generated new TSK 0
I20260811 10:16:06.492358 5727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:16:06.530565 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:06.615764 5753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:06.615996 5753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:06.616034 5753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:06.617169 5753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:06.617235 5753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:06.617281 5753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:06.618737 5753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:06.619000 5753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:06.619223 5753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:06.624483 5765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.624450 5772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.624502 5753 server_base.cc:1034] running on GCE node
W20260811 10:16:06.624466 5767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.624873 5753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:06.625145 5753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:06.626314 5753 hybrid_clock.cc:648] HybridClock initialized: now 1786443366626294 us; error 38 us; skew 500 ppm
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:06.629292 5753 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:16:06.629752 5753 webserver.cc:460] Webserver started at http://127.8.238.1:45409/ using document root <none> and password file <none>
I20260811 10:16:06.630004 5753 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:06.630051 5753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:06.630184 5753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:06.631225 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "a3849b76f0ea48b7abe0c76f49898dd4"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "6647dd348c82711b858f125b1a9faab3"
server_key_iv: "bf1fe0945ad7921e2bf613631bb55bb2"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.631597 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "a3849b76f0ea48b7abe0c76f49898dd4"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "6647dd348c82711b858f125b1a9faab3"
server_key_iv: "bf1fe0945ad7921e2bf613631bb55bb2"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.632998 5753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:06.633819 5786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.634020 5753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:06.634091 5753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "a3849b76f0ea48b7abe0c76f49898dd4"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "6647dd348c82711b858f125b1a9faab3"
server_key_iv: "bf1fe0945ad7921e2bf613631bb55bb2"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.634176 5753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:06.652995 5753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:06.654942 5753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:06.655071 5753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:06.655278 5753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:06.655620 5753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:06.655681 5753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.655717 5753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:06.655786 5753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.662580 5753 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:32921
I20260811 10:16:06.662642 5996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:32921 every 8 connection(s)
I20260811 10:16:06.662976 5753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:16:06.666867 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 5753
I20260811 10:16:06.666976 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:06.667171 9144 external_mini_cluster.cc:1428] Setting key 4c6df71ea6a85b31afa5387130b58099
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:06.673136 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.663809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:49447 (local address 127.8.238.62:33579)
0811 10:16:06.663892 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:06.663894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:06.664091 (+ 197us) server_negotiation.cc:407] Connection header received
0811 10:16:06.664599 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:06.664600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:06.664611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:06.664665 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:06.665635 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.666147 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.667487 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.667801 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.667862 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.670109 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:06.670111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:06.670111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:06.670121 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:06.671614 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.672064 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.672065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.672066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.672132 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.672398 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.672399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.672400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.672574 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:06.672657 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:06.672945 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 10:16:06.673042 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:16:06.673326 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.663738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33579 (local address 127.8.238.1:49447)
0811 10:16:06.663960 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:06.664000 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:06.664513 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:06.664721 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:06.664724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:06.665335 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:06.665556 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.665563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.666230 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.666231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.667367 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.667371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.667903 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.667904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.668057 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.668597 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:06.668606 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:06.670003 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:06.671720 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.671728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.671731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.671982 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.672204 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.672205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.672206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.672322 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.672661 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:06.672669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:06.672807 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 10:16:06.672922 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 10:16:06.673085 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260811 10:16:06.673871 5997 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33579
I20260811 10:16:06.673981 5997 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:06.674162 5997 heartbeater.cc:511] Master 127.8.238.62:33579 requested a full tablet report, sending...
I20260811 10:16:06.674734 5645 ts_manager.cc:194] Registered new tserver with Master: a3849b76f0ea48b7abe0c76f49898dd4 (127.8.238.1:32921)
I20260811 10:16:06.675349 5645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:49447
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:16:06.696775 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:06.782394 6015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:06.782611 6015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:06.782635 6015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:06.783806 6015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:06.783850 6015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:06.783910 6015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:06.785372 6015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:06.785638 6015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:06.785892 6015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:06.791124 6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.791132 6021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.791132 6028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.791507 6015 server_base.cc:1034] running on GCE node
I20260811 10:16:06.791783 6015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:06.792043 6015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:06.793220 6015 hybrid_clock.cc:648] HybridClock initialized: now 1786443366793206 us; error 32 us; skew 500 ppm
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:06.795915 6015 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:16:06.796415 6015 webserver.cc:460] Webserver started at http://127.8.238.2:38319/ using document root <none> and password file <none>
I20260811 10:16:06.796685 6015 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:06.796757 6015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:06.796904 6015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:06.797957 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "118b2f00abe84b47be45f31a7c9a97c6"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "bf173367f8e4cc71c51aaaac67aa42c9"
server_key_iv: "e3e89c16bd0427462c9292709e255924"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.798337 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "118b2f00abe84b47be45f31a7c9a97c6"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "bf173367f8e4cc71c51aaaac67aa42c9"
server_key_iv: "e3e89c16bd0427462c9292709e255924"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.799687 6015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:16:06.800458 6041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.800628 6015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:16:06.800743 6015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "118b2f00abe84b47be45f31a7c9a97c6"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "bf173367f8e4cc71c51aaaac67aa42c9"
server_key_iv: "e3e89c16bd0427462c9292709e255924"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.800820 6015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:06.820861 6015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:06.823179 6015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:06.823378 6015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:06.823637 6015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:06.824019 6015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:06.824060 6015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.824110 6015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:06.824142 6015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.830822 6015 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:37521
I20260811 10:16:06.830892 6237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:37521 every 8 connection(s)
I20260811 10:16:06.831280 6015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:16:06.832764 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 6015
I20260811 10:16:06.832916 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:06.833132 9144 external_mini_cluster.cc:1428] Setting key 953d194dd2cee65bef3080864d8068e3
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:06.841918 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.832154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:58043 (local address 127.8.238.62:33579)
0811 10:16:06.832223 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:06.832225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:06.832455 (+ 230us) server_negotiation.cc:407] Connection header received
0811 10:16:06.833077 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:06.833079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:06.833093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:06.833146 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:06.834327 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.834847 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.835957 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.836172 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.836253 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.838916 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:06.838918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:06.838918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:06.838926 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:06.840450 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.840888 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.840889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.840890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.840958 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.841228 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.841229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.841229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.841358 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:06.841433 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:06.841728 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 10:16:06.841821 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 10:16:06.842136 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.832118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33579 (local address 127.8.238.2:58043)
0811 10:16:06.832322 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:06.832360 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:06.832967 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:06.833216 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:06.833221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:06.834015 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:06.834229 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.834234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.834959 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.834961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.835870 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.835873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.836275 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.836277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.836407 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.837032 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:06.837041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:06.838812 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:06.840563 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.840572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.840575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.840807 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.841036 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.841037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.841037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.841157 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.841436 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:06.841445 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:06.841596 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 10:16:06.841704 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 10:16:06.841886 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260811 10:16:06.842660 6238 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33579
I20260811 10:16:06.842783 6238 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:06.842971 6238 heartbeater.cc:511] Master 127.8.238.62:33579 requested a full tablet report, sending...
I20260811 10:16:06.843420 5645 ts_manager.cc:194] Registered new tserver with Master: 118b2f00abe84b47be45f31a7c9a97c6 (127.8.238.2:37521)
I20260811 10:16:06.843884 5645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:58043
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:16:06.863366 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:40007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:06.949903 6244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:06.950125 6244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:06.950148 6244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:06.951238 6244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:06.951274 6244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:06.951300 6244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:06.952857 6244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:40007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:06.953047 6244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:06.953230 6244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:06.958091 6252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:06.958135 6250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.958343 6244 server_base.cc:1034] running on GCE node
W20260811 10:16:06.958093 6254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:06.958596 6244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:06.958815 6244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:06.959957 6244 hybrid_clock.cc:648] HybridClock initialized: now 1786443366959936 us; error 40 us; skew 500 ppm
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:06.962589 6244 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:16:06.963038 6244 webserver.cc:460] Webserver started at http://127.8.238.3:33493/ using document root <none> and password file <none>
I20260811 10:16:06.963230 6244 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:06.963270 6244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:06.963357 6244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:06.964411 6244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "d867081c71ac4d6781fccb9d0fbf14cf"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "b8f2ad1969f36f7272ee5a40f5eaf683"
server_key_iv: "625d5e62d27c0f86e425fdc15da10662"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.964721 6244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "d867081c71ac4d6781fccb9d0fbf14cf"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "b8f2ad1969f36f7272ee5a40f5eaf683"
server_key_iv: "625d5e62d27c0f86e425fdc15da10662"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.966019 6244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:06.966732 6266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.966961 6244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:06.967053 6244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "d867081c71ac4d6781fccb9d0fbf14cf"
format_stamp: "Formatted at 2026-08-11 10:16:06 on dist-test-slave-7f01"
server_key: "b8f2ad1969f36f7272ee5a40f5eaf683"
server_key_iv: "625d5e62d27c0f86e425fdc15da10662"
server_key_version: "encryptionkey@0"
I20260811 10:16:06.967106 6244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:06.975447 6244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:06.977658 6244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:06.977766 6244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:06.977978 6244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:06.978291 6244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:06.978327 6244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.978349 6244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:06.978401 6244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:06.984526 6244 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:40557
I20260811 10:16:06.984622 6466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:40557 every 8 connection(s)
I20260811 10:16:06.984925 6244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:16:06.989012 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 6244
I20260811 10:16:06.989135 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:06.989306 9144 external_mini_cluster.cc:1428] Setting key 92d8873343d9455858c4706adfc0dca9
Aug 11 10:16:06 dist-test-slave-7f01 krb5kdc[5551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:06.994477 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.985683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:44543 (local address 127.8.238.62:33579)
0811 10:16:06.985747 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:06.985750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:06.985953 (+ 203us) server_negotiation.cc:407] Connection header received
0811 10:16:06.986413 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:06.986414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:06.986424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:06.986482 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:06.987468 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.988058 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.989113 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.989368 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.989453 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.991866 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:06.991868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:06.991868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:06.991874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:06.993349 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.993710 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.993711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.993712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.993764 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:06.993974 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:06.993975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:06.993975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:06.994073 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:06.994123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:06.994352 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 10:16:06.994413 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:16:06.994666 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:06.985631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:33579 (local address 127.8.238.3:44543)
0811 10:16:06.985807 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:06.985855 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:06.986325 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:06.986554 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:06.986557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:06.987157 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:06.987348 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.987375 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.988172 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.988173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.988984 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:06.988987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:06.989464 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:06.989465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:06.989617 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:06.990179 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:06.990190 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:06.991724 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:06.993424 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.993430 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.993432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.993648 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.993821 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:06.993822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:06.993822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:06.993922 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:06.994128 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:06.994133 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:06.994239 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 10:16:06.994318 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 10:16:06.994441 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260811 10:16:06.995122 6468 heartbeater.cc:347] Connected to a master server at 127.8.238.62:33579
I20260811 10:16:06.995217 6468 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:06.995357 6468 heartbeater.cc:511] Master 127.8.238.62:33579 requested a full tablet report, sending...
I20260811 10:16:06.995728 5645 ts_manager.cc:194] Registered new tserver with Master: d867081c71ac4d6781fccb9d0fbf14cf (127.8.238.3:40557)
I20260811 10:16:06.996112 5645 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:44543
I20260811 10:16:07.001401 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:07.008261 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.002703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.8.238.62:33579)
0811 10:16:07.002770 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.002772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.002854 (+ 82us) server_negotiation.cc:407] Connection header received
0811 10:16:07.002903 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.002904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.002912 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.002973 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:07.003479 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.004019 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.004703 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.004941 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.004988 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.006100 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:07.006101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:07.006102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:07.006108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:07.007272 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:07.007578 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:07.007579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:07.007579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:07.007638 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:07.007826 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:07.007827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:07.007827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:07.007966 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:07.008043 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.008125 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.008180 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 10:16:07.009853 5645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:07.011472 5645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:07.018249 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.014254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.8.238.2:37521)
0811 10:16:07.014477 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.014482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.014494 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:07.014604 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.014606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.014624 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.014699 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:07.014970 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.016147 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.017335 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.017840 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.017936 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018001 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.018117 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.018157 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260811 10:16:07.018297 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.013920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32921 (local address 127.0.0.1:40390)
0811 10:16:07.014032 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.014074 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.014155 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.014777 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.014780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.014782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:07.014882 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.014884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.015853 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.015854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.017035 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.017037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.018023 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.018024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.018088 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018093 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.018127 (+ 34us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.018180 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":5,"thread_start_us":41,"threads_started":1}
I20260811 10:16:07.018412 6503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.014284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:40557 (local address 127.0.0.1:40666)
0811 10:16:07.014430 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.014441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.014495 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.014778 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.014786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.014789 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:07.014865 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.014867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.016231 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.016232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.017318 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.017322 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.018205 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.018206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.018267 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018271 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.018300 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.018340 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":21,"thread_start_us":92,"threads_started":1}
I20260811 10:16:07.018266 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.014102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:37521 (local address 127.0.0.1:53746)
0811 10:16:07.014417 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.014429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.014483 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.014782 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.014783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.014785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:07.014882 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.014885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.016254 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.016255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.017243 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.017246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.017935 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.017937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.018016 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018030 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.018103 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.018155 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":24,"thread_start_us":81,"threads_started":1}
I20260811 10:16:07.018671 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.014104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.8.238.1:32921)
0811 10:16:07.014595 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.014599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.014608 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:16:07.014650 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.014652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.014667 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.014968 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:07.015053 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.015720 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.017130 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.017915 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.018406 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018485 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.018517 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.018569 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":87,"threads_started":1}
I20260811 10:16:07.018874 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.014399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.8.238.3:40557)
0811 10:16:07.014610 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.014615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.014625 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:07.014662 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.014665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.014679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.014967 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:07.015039 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.016112 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.017407 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.018116 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.018642 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.018704 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.018741 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.018785 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":90,"threads_started":1}
I20260811 10:16:07.019345 5874 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 (DEFAULT_TABLE table=test-table [id=69794120ea494058b8d22ffad570fce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:07.019354 6134 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 (DEFAULT_TABLE table=test-table [id=69794120ea494058b8d22ffad570fce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:07.019445 6362 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 (DEFAULT_TABLE table=test-table [id=69794120ea494058b8d22ffad570fce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:07.019695 6362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:07.019701 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:07.019701 5874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:07.022493 6511 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Bootstrap starting.
I20260811 10:16:07.023097 6511 tablet_bootstrap.cc:654] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:07.023222 6512 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Bootstrap starting.
I20260811 10:16:07.023545 6513 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Bootstrap starting.
I20260811 10:16:07.023622 6511 log.cc:824] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:07.023921 6512 tablet_bootstrap.cc:654] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:07.024333 6513 tablet_bootstrap.cc:654] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:07.024442 6511 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: No bootstrap required, opened a new log
I20260811 10:16:07.024523 6511 ts_tablet_manager.cc:1397] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:07.024569 6512 log.cc:824] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:07.024955 6513 log.cc:824] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:07.025476 6512 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: No bootstrap required, opened a new log
I20260811 10:16:07.025552 6512 ts_tablet_manager.cc:1397] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:07.025682 6513 tablet_bootstrap.cc:492] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: No bootstrap required, opened a new log
I20260811 10:16:07.025746 6513 ts_tablet_manager.cc:1397] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:07.026400 6511 raft_consensus.cc:348] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.026562 6511 raft_consensus.cc:374] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:07.026609 6511 raft_consensus.cc:729] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d867081c71ac4d6781fccb9d0fbf14cf, State: Initialized, Role: FOLLOWER
I20260811 10:16:07.026726 6511 consensus_queue.cc:260] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.027038 6511 ts_tablet_manager.cc:1428] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:07.027177 6468 heartbeater.cc:503] Master 127.8.238.62:33579 was elected leader, sending a full tablet report...
I20260811 10:16:07.027546 6512 raft_consensus.cc:348] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.027724 6512 raft_consensus.cc:374] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:07.027796 6512 raft_consensus.cc:729] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3849b76f0ea48b7abe0c76f49898dd4, State: Initialized, Role: FOLLOWER
I20260811 10:16:07.027662 6513 raft_consensus.cc:348] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.027865 6513 raft_consensus.cc:374] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:07.027915 6513 raft_consensus.cc:729] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118b2f00abe84b47be45f31a7c9a97c6, State: Initialized, Role: FOLLOWER
I20260811 10:16:07.027947 6512 consensus_queue.cc:260] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.028008 6513 consensus_queue.cc:260] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.028229 6512 ts_tablet_manager.cc:1428] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 10:16:07.028265 6513 ts_tablet_manager.cc:1428] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:07.028234 5997 heartbeater.cc:503] Master 127.8.238.62:33579 was elected leader, sending a full tablet report...
I20260811 10:16:07.028357 6238 heartbeater.cc:503] Master 127.8.238.62:33579 was elected leader, sending a full tablet report...
W20260811 10:16:07.081918 6239 tablet.cc:2369] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:16:07.163877 5998 tablet.cc:2369] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:16:07.235515 6469 tablet.cc:2369] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:07.290868 6522 raft_consensus.cc:482] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:07.291049 6522 raft_consensus.cc:504] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.291402 6522 leader_election.cc:288] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3849b76f0ea48b7abe0c76f49898dd4 (127.8.238.1:32921), 118b2f00abe84b47be45f31a7c9a97c6 (127.8.238.2:37521)
I20260811 10:16:07.294399 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.291731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:49691 (local address 127.8.238.1:32921)
0811 10:16:07.291796 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.291798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.291809 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:07.291903 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.291905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.291922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.292016 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:07.292180 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.292687 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.293630 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.294030 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.294114 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.294127 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.294287 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.294328 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:16:07.294399 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.291629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32921 (local address 127.8.238.3:49691)
0811 10:16:07.291699 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.291714 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.291802 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.292021 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.292023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.292025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:07.292104 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.292107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.292775 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.292776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.293534 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.293536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.294127 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.294128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.294211 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.294214 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.294286 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.294327 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 10:16:07.294513 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.291743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:37521 (local address 127.8.238.3:51583)
0811 10:16:07.291937 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.291946 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.292027 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.292189 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.292190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.292192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:07.292271 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.292272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.292924 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.292925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.293652 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.293653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.294248 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.294249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.294322 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.294325 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.294391 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.294430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260811 10:16:07.294508 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.291879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:51583 (local address 127.8.238.2:37521)
0811 10:16:07.291932 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.291934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.292021 (+ 87us) server_negotiation.cc:407] Connection header received
0811 10:16:07.292096 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.292098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.292116 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.292172 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:07.292342 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.292826 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.293742 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.294146 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.294229 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.294241 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.294398 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.294437 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:16:07.294835 5919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd9cf8349fa4ed2bcbe65f4fe9a0669" candidate_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" is_pre_election: true
I20260811 10:16:07.294893 6161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd9cf8349fa4ed2bcbe65f4fe9a0669" candidate_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b2f00abe84b47be45f31a7c9a97c6" is_pre_election: true
I20260811 10:16:07.295009 5919 raft_consensus.cc:2454] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d867081c71ac4d6781fccb9d0fbf14cf in term 0.
I20260811 10:16:07.295058 6161 raft_consensus.cc:2454] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d867081c71ac4d6781fccb9d0fbf14cf in term 0.
I20260811 10:16:07.295238 6281 leader_election.cc:302] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3849b76f0ea48b7abe0c76f49898dd4, d867081c71ac4d6781fccb9d0fbf14cf; no voters:
I20260811 10:16:07.295379 6522 raft_consensus.cc:2781] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:07.295441 6522 raft_consensus.cc:482] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:07.295480 6522 raft_consensus.cc:3037] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:07.296280 6522 raft_consensus.cc:504] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.296414 6522 leader_election.cc:288] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [CANDIDATE]: Term 1 election: Requested vote from peers a3849b76f0ea48b7abe0c76f49898dd4 (127.8.238.1:32921), 118b2f00abe84b47be45f31a7c9a97c6 (127.8.238.2:37521)
I20260811 10:16:07.296576 5919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd9cf8349fa4ed2bcbe65f4fe9a0669" candidate_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3849b76f0ea48b7abe0c76f49898dd4"
I20260811 10:16:07.296586 6161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd9cf8349fa4ed2bcbe65f4fe9a0669" candidate_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118b2f00abe84b47be45f31a7c9a97c6"
I20260811 10:16:07.296645 5919 raft_consensus.cc:3037] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:07.296693 6161 raft_consensus.cc:3037] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:07.297480 5919 raft_consensus.cc:2454] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d867081c71ac4d6781fccb9d0fbf14cf in term 1.
I20260811 10:16:07.297495 6161 raft_consensus.cc:2454] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d867081c71ac4d6781fccb9d0fbf14cf in term 1.
I20260811 10:16:07.297657 6281 leader_election.cc:302] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118b2f00abe84b47be45f31a7c9a97c6, d867081c71ac4d6781fccb9d0fbf14cf; no voters:
I20260811 10:16:07.297768 6522 raft_consensus.cc:2781] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:07.297957 6522 raft_consensus.cc:686] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 1 LEADER]: Becoming Leader. State: Replica: d867081c71ac4d6781fccb9d0fbf14cf, State: Running, Role: LEADER
I20260811 10:16:07.298061 6522 consensus_queue.cc:237] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } }
I20260811 10:16:07.298894 5643 catalog_manager.cc:5495] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf reported cstate change: term changed from 0 to 1, leader changed from <none> to d867081c71ac4d6781fccb9d0fbf14cf (127.8.238.3). New cstate: current_term: 1 leader_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d867081c71ac4d6781fccb9d0fbf14cf" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 40557 } health_report { overall_health: HEALTHY } } }
I20260811 10:16:07.369657 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.367209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.8.238.3:40557)
0811 10:16:07.367281 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.367283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.367291 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:16:07.367335 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.367335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.367358 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.367428 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:07.367546 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.368057 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.368831 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.369036 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.369103 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.369278 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:07.369446 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:07.369524 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.369568 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.369592 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:16:07.371841 6161 raft_consensus.cc:1261] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 1 FOLLOWER]: Refusing update from remote peer d867081c71ac4d6781fccb9d0fbf14cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:07.371897 5919 raft_consensus.cc:1261] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P a3849b76f0ea48b7abe0c76f49898dd4 [term 1 FOLLOWER]: Refusing update from remote peer d867081c71ac4d6781fccb9d0fbf14cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:07.372258 6522 consensus_queue.cc:1035] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "118b2f00abe84b47be45f31a7c9a97c6" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:07.372423 6528 consensus_queue.cc:1035] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3849b76f0ea48b7abe0c76f49898dd4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 32921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:07.375926 6533 mvcc.cc:205] Tried to move back new op lower bound from 7317272032751616000 to 7317272032453427200. Current Snapshot: MvccSnapshot[applied={T|T < 7317272032751616000}]
I20260811 10:16:07.377601 5644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:07.377710 5644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:07.378556 5644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25e20489dbaa461b9343e6b299104e60: Sending DeleteTablet for 3 replicas of tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669
I20260811 10:16:07.378880 5874 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:07 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:40390
I20260811 10:16:07.378998 6134 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:07 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:53746
I20260811 10:16:07.379087 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 5753
I20260811 10:16:07.379223 6362 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:07 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:40666
I20260811 10:16:07.383581 6553 tablet_replica.cc:330] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: stopping tablet replica
I20260811 10:16:07.383675 6553 raft_consensus.cc:2229] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:16:07.383780 6553 raft_consensus.cc:2258] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:07.395905 6552 tablet_replica.cc:330] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: stopping tablet replica
I20260811 10:16:07.396181 6552 raft_consensus.cc:2229] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:07.396452 6552 raft_consensus.cc:2258] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:07.398062 6552 ts_tablet_manager.cc:1905] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:07.400827 6552 ts_tablet_manager.cc:1918] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:07.401005 6552 log.cc:1196] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/2fd9cf8349fa4ed2bcbe65f4fe9a0669
I20260811 10:16:07.401468 6552 ts_tablet_manager.cc:1939] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P 118b2f00abe84b47be45f31a7c9a97c6: Deleting consensus metadata
I20260811 10:16:07.402148 5616 catalog_manager.cc:4841] TS 118b2f00abe84b47be45f31a7c9a97c6 (127.8.238.2:37521): tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 (table test-table [id=69794120ea494058b8d22ffad570fce3]) successfully deleted
I20260811 10:16:07.407227 6553 ts_tablet_manager.cc:1905] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:07.408943 6553 ts_tablet_manager.cc:1918] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:07.409030 6553 log.cc:1196] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/2fd9cf8349fa4ed2bcbe65f4fe9a0669
I20260811 10:16:07.409313 6553 ts_tablet_manager.cc:1939] T 2fd9cf8349fa4ed2bcbe65f4fe9a0669 P d867081c71ac4d6781fccb9d0fbf14cf: Deleting consensus metadata
I20260811 10:16:07.409827 5616 catalog_manager.cc:4841] TS d867081c71ac4d6781fccb9d0fbf14cf (127.8.238.3:40557): tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669 (table test-table [id=69794120ea494058b8d22ffad570fce3]) successfully deleted
W20260811 10:16:07.412113 5616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:32921 (error 108)
I20260811 10:16:07.412458 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 6015
I20260811 10:16:07.419445 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.419282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:32921 (local address 127.0.0.1:34124)
0811 10:16:07.419348 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.419397 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:32921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 10:16:07.420743 5616 catalog_manager.cc:4584] TS a3849b76f0ea48b7abe0c76f49898dd4 (127.8.238.1:32921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fd9cf8349fa4ed2bcbe65f4fe9a0669: Network error: Client connection negotiation failed: client connection to 127.8.238.1:32921: connect: Connection refused (error 111)
I20260811 10:16:07.424911 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 6244
I20260811 10:16:07.434079 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 5580
2026-08-11T10:16:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2506 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T10:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:07Z Disabled control of system clock
I20260811 10:16:07.467939 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39251
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:37375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:07.549641 6559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:07.549861 6559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:07.549881 6559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:07.550947 6559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:07.550980 6559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:07.550993 6559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:07.551007 6559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:07.551018 6559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:07.552277 6559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:07.552459 6559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:07.552700 6559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:07.557274 6573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.557282 6571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.557252 6570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:07.557319 6559 server_base.cc:1034] running on GCE node
I20260811 10:16:07.557766 6559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:07.558061 6559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:07.559231 6559 hybrid_clock.cc:648] HybridClock initialized: now 1786443367559218 us; error 45 us; skew 500 ppm
I20260811 10:16:07.560583 6559 webserver.cc:460] Webserver started at http://127.8.238.62:45215/ using document root <none> and password file <none>
I20260811 10:16:07.560828 6559 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:07.560901 6559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:07.561049 6559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:07.562075 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "91d78072d16d4ecaa607d2fc665877a7"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "ae54e88e939b965408c730b6a7e9b442"
server_key_iv: "25bedd1ce90e2b809ba784982c57aad1"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.562444 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "91d78072d16d4ecaa607d2fc665877a7"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "ae54e88e939b965408c730b6a7e9b442"
server_key_iv: "25bedd1ce90e2b809ba784982c57aad1"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.563834 6559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:16:07.564611 6580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.564800 6559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:07.564906 6559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "91d78072d16d4ecaa607d2fc665877a7"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "ae54e88e939b965408c730b6a7e9b442"
server_key_iv: "25bedd1ce90e2b809ba784982c57aad1"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.564993 6559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:07.575187 6559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:07.575479 6559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:07.575639 6559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:07.579890 6559 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39251
I20260811 10:16:07.579924 6684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39251 every 8 connection(s)
I20260811 10:16:07.580251 6559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:07.581190 6686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:07.582244 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 6559
I20260811 10:16:07.582351 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:07.582571 9144 external_mini_cluster.cc:1428] Setting key 847ec2a4b9b1bc7e22ed1a9c8dc39e68
I20260811 10:16:07.584052 6686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: Bootstrap starting.
I20260811 10:16:07.584801 6686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:07.585153 6686 log.cc:824] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:07.585898 6686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: No bootstrap required, opened a new log
I20260811 10:16:07.588194 6686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } }
I20260811 10:16:07.588402 6686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:07.588456 6686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d78072d16d4ecaa607d2fc665877a7, State: Initialized, Role: FOLLOWER
I20260811 10:16:07.588603 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.583124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.8.238.62:39251)
0811 10:16:07.583346 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.583354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.583387 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:16:07.583954 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.583963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.584038 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.584182 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:07.584409 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.585666 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.586470 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.587091 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.587225 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.587893 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:07.587898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:07.587899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:07.588099 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:07.588140 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:07.588154 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:07.588221 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.588287 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.588363 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260811 10:16:07.588587 6686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } }
I20260811 10:16:07.588670 6686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:07.588707 6686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:07.588748 6686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:07.589257 6686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } }
I20260811 10:16:07.589380 6686 leader_election.cc:302] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91d78072d16d4ecaa607d2fc665877a7; no voters:
I20260811 10:16:07.589560 6686 leader_election.cc:288] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:07.589592 6692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:07.589766 6692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [term 1 LEADER]: Becoming Leader. State: Replica: 91d78072d16d4ecaa607d2fc665877a7, State: Running, Role: LEADER
I20260811 10:16:07.589819 6686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:07.589903 6692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } }
I20260811 10:16:07.590287 6695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d78072d16d4ecaa607d2fc665877a7. Latest consensus state: current_term: 1 leader_uuid: "91d78072d16d4ecaa607d2fc665877a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } } }
I20260811 10:16:07.590277 6693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91d78072d16d4ecaa607d2fc665877a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d78072d16d4ecaa607d2fc665877a7" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39251 } } }
I20260811 10:16:07.590381 6695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:07.590389 6693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:07.590914 6708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:07.591373 6708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:07.592952 6708 catalog_manager.cc:1347] Generated new cluster ID: c8f406dbd71047c1a4ce0d580429bcd8
I20260811 10:16:07.593012 6708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:07.606348 6708 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:07.606870 6708 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:07.610909 6708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: Generated new TSK 0
I20260811 10:16:07.611090 6708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:07.616621 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--builtin_ntp_servers=127.8.238.20:37375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:07.708408 6734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:07.708628 6734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:07.708653 6734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:07.709910 6734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:07.709954 6734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:07.709981 6734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:07.711444 6734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:07.711700 6734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:07.712076 6734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:07.717536 6745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.717537 6741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.717563 6753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:07.717813 6734 server_base.cc:1034] running on GCE node
I20260811 10:16:07.718118 6734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:07.718458 6734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:07.719623 6734 hybrid_clock.cc:648] HybridClock initialized: now 1786443367719612 us; error 30 us; skew 500 ppm
I20260811 10:16:07.721171 6734 webserver.cc:460] Webserver started at http://127.8.238.1:42991/ using document root <none> and password file <none>
I20260811 10:16:07.721462 6734 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:07.721516 6734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:07.721715 6734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:07.722954 6734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e01ba5bc74654f4f12bba3a5eb272dd3"
server_key_iv: "c1a7676847cea60d1aed54a42137b221"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.723367 6734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e01ba5bc74654f4f12bba3a5eb272dd3"
server_key_iv: "c1a7676847cea60d1aed54a42137b221"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.724876 6734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:16:07.725834 6762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.726018 6734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 10:16:07.726135 6734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e01ba5bc74654f4f12bba3a5eb272dd3"
server_key_iv: "c1a7676847cea60d1aed54a42137b221"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.726224 6734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:07.739535 6734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:07.739892 6734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:07.740047 6734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:07.740280 6734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:07.740607 6734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:07.740664 6734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.740705 6734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:07.740732 6734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.746978 6734 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:34541
I20260811 10:16:07.747032 6967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:34541 every 8 connection(s)
I20260811 10:16:07.747346 6734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:16:07.752748 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 6734
I20260811 10:16:07.752863 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:07.752950 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.748101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:44761 (local address 127.8.238.62:39251)
0811 10:16:07.748351 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.748354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.748544 (+ 190us) server_negotiation.cc:407] Connection header received
0811 10:16:07.749112 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.749114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.749129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.749202 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:07.749917 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.750438 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.751432 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.751637 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.751715 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.752624 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:07.752625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:07.752626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:07.752631 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:07.752648 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:07.752658 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:07.752719 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.752839 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.752867 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260811 10:16:07.753108 9144 external_mini_cluster.cc:1428] Setting key ca318f965e4f65653891898fc10d07f9
I20260811 10:16:07.753168 6970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.748053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39251 (local address 127.8.238.1:44761)
0811 10:16:07.748409 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.748454 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.748991 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.749290 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.749295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.749312 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:07.749821 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.749826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.750539 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.750541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.751338 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.751340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.751727 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.751728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.751896 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.752446 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:07.752447 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:07.752474 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:07.752498 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:07.752721 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:07.752728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:07.752732 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.752838 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.752940 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":101,"threads_started":1}
I20260811 10:16:07.753697 6968 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39251
I20260811 10:16:07.753818 6968 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:07.754082 6968 heartbeater.cc:511] Master 127.8.238.62:39251 requested a full tablet report, sending...
I20260811 10:16:07.754164 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--builtin_ntp_servers=127.8.238.20:37375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:16:07.754596 6619 ts_manager.cc:194] Registered new tserver with Master: 1e59251cd8d04b30b84a1f7f0dfe5f31 (127.8.238.1:34541)
I20260811 10:16:07.755221 6619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.1:44761
W20260811 10:16:07.837611 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:07.837812 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:07.837836 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:07.838879 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:07.838919 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:07.838944 6971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:07.840308 6971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:07.840493 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:07.840734 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:07.845695 6983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.845703 6980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.845690 6986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:07.845875 6971 server_base.cc:1034] running on GCE node
I20260811 10:16:07.846131 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:07.846431 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:07.847602 6971 hybrid_clock.cc:648] HybridClock initialized: now 1786443367847574 us; error 42 us; skew 500 ppm
I20260811 10:16:07.849103 6971 webserver.cc:460] Webserver started at http://127.8.238.2:34781/ using document root <none> and password file <none>
I20260811 10:16:07.849385 6971 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:07.849438 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:07.849628 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:07.850785 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "6d612db398434d8187f89e7fade415a1"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e9c3909821e5e54f5e5769493c798b68"
server_key_iv: "df2a299f93eb149ce42cb67d4a5dffb0"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.851112 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "6d612db398434d8187f89e7fade415a1"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e9c3909821e5e54f5e5769493c798b68"
server_key_iv: "df2a299f93eb149ce42cb67d4a5dffb0"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.852559 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:16:07.853322 6996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.853516 6971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:07.853622 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "6d612db398434d8187f89e7fade415a1"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e9c3909821e5e54f5e5769493c798b68"
server_key_iv: "df2a299f93eb149ce42cb67d4a5dffb0"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.853703 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:07.867654 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:07.868058 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:07.868234 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:07.868438 6971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:07.868798 6971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:07.868865 6971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.868917 6971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:07.868944 6971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.875545 6971 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:45503
I20260811 10:16:07.875598 7214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:45503 every 8 connection(s)
I20260811 10:16:07.876014 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:16:07.880261 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 6971
I20260811 10:16:07.880363 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:07.880579 9144 external_mini_cluster.cc:1428] Setting key c3e9bab20bcfcf65747d43631653a142
I20260811 10:16:07.881983 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--builtin_ntp_servers=127.8.238.20:37375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:16:07.885615 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.876852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:53741 (local address 127.8.238.62:39251)
0811 10:16:07.876911 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:07.876915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:07.880034 (+ 3119us) server_negotiation.cc:407] Connection header received
0811 10:16:07.880810 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:07.880812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:07.880829 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:07.880922 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:07.881926 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.882735 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.884106 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.884321 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.884405 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.885306 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:07.885307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:07.885308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:07.885313 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:07.885330 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:07.885334 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:07.885403 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:07.885504 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 10:16:07.885534 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:16:07.885840 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:07.876824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39251 (local address 127.8.238.2:53741)
0811 10:16:07.879888 (+ 3064us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:07.879924 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:07.880665 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:07.881042 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:07.881046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:07.881078 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:07.881809 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.881817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.882870 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.882871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.884001 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:07.884004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:07.884426 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:07.884427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:07.884562 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:07.885163 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:07.885164 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:07.885193 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:07.885216 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:07.885408 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:07.885415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:07.885418 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:07.885506 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 10:16:07.885610 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3002,"thread_start_us":126,"threads_started":1}
I20260811 10:16:07.886443 7216 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39251
I20260811 10:16:07.886559 7216 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:07.886742 7216 heartbeater.cc:511] Master 127.8.238.62:39251 requested a full tablet report, sending...
I20260811 10:16:07.887226 6619 ts_manager.cc:194] Registered new tserver with Master: 6d612db398434d8187f89e7fade415a1 (127.8.238.2:45503)
I20260811 10:16:07.887880 6619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.2:53741
W20260811 10:16:07.969249 7224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:07.969449 7224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:07.969470 7224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:07.970535 7224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:07.970567 7224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:07.970592 7224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:07.971921 7224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:07.972106 7224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:07.972340 7224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:07.977310 7234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.977454 7236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:07.977525 7231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:07.977901 7224 server_base.cc:1034] running on GCE node
I20260811 10:16:07.978140 7224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:07.978379 7224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:07.979560 7224 hybrid_clock.cc:648] HybridClock initialized: now 1786443367979544 us; error 27 us; skew 500 ppm
I20260811 10:16:07.980866 7224 webserver.cc:460] Webserver started at http://127.8.238.3:44551/ using document root <none> and password file <none>
I20260811 10:16:07.981134 7224 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:07.981209 7224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:07.981360 7224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:07.982445 7224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e1a50a0bfbea44d61b9d6a112b624f85"
server_key_iv: "55d7cd2aa4ac83a01ad5674dbb689f9c"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.982821 7224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e1a50a0bfbea44d61b9d6a112b624f85"
server_key_iv: "55d7cd2aa4ac83a01ad5674dbb689f9c"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.984236 7224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 10:16:07.985050 7249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:07.985239 7224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:07.985352 7224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26"
format_stamp: "Formatted at 2026-08-11 10:16:07 on dist-test-slave-7f01"
server_key: "e1a50a0bfbea44d61b9d6a112b624f85"
server_key_iv: "55d7cd2aa4ac83a01ad5674dbb689f9c"
server_key_version: "encryptionkey@0"
I20260811 10:16:07.985435 7224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:08.002971 7224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:08.003271 7224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:08.003423 7224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:08.003650 7224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:08.004032 7224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:08.004092 7224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:08.004127 7224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:08.004154 7224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:08.010039 7224 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:35953
I20260811 10:16:08.010114 7460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:35953 every 8 connection(s)
I20260811 10:16:08.010449 7224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:16:08.015472 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.011176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:35755 (local address 127.8.238.62:39251)
0811 10:16:08.011270 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.011273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.011485 (+ 212us) server_negotiation.cc:407] Connection header received
0811 10:16:08.012068 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.012069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.012080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.012133 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:08.012792 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.013284 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.014216 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.014424 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.014479 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.015235 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:08.015236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:08.015236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:08.015240 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:08.015254 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:08.015258 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:08.015303 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.015388 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.015409 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 10:16:08.015713 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.011147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39251 (local address 127.8.238.3:35755)
0811 10:16:08.011361 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.011394 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.011976 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.012185 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.012188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.012196 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:08.012693 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.012697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.013370 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.013371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.014143 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.014145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.014509 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.014510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.014624 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.015107 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:08.015108 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:08.015135 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:08.015157 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:08.015315 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:08.015320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:08.015322 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.015378 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.015460 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260811 10:16:08.016312 7461 heartbeater.cc:347] Connected to a master server at 127.8.238.62:39251
I20260811 10:16:08.016418 7461 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:08.016603 7461 heartbeater.cc:511] Master 127.8.238.62:39251 requested a full tablet report, sending...
I20260811 10:16:08.016991 6619 ts_manager.cc:194] Registered new tserver with Master: 842c2fdfb4d949e8b6b5c8f906bc3d26 (127.8.238.3:35953)
I20260811 10:16:08.017376 6619 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.3:35755
I20260811 10:16:08.018791 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 7224
I20260811 10:16:08.018872 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:08.019027 9144 external_mini_cluster.cc:1428] Setting key cb8f2021d1c06efc31b7403b014865af
I20260811 10:16:08.019896 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:08.023903 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.021231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.8.238.62:39251)
0811 10:16:08.021286 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.021288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.021417 (+ 129us) server_negotiation.cc:407] Connection header received
0811 10:16:08.021475 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.021476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.021484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.021536 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:08.021670 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.022127 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.022831 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.023013 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.023074 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.023633 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:08.023635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:08.023635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:08.023639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:08.023652 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:08.023655 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:08.023699 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.023824 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.023846 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:16:08.025441 6619 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:08.027009 6619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:08.033249 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34541 (local address 127.0.0.1:46204)
0811 10:16:08.029572 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.029582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.029624 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.029933 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.029934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.029937 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:08.029983 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.029985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.030996 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.030997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.031964 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.031967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.032934 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.032936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.033015 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033029 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.033113 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.033167 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":98,"threads_started":1}
I20260811 10:16:08.033265 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.8.238.1:34541)
0811 10:16:08.029776 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.029779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.029791 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:08.029839 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.029841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.029857 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.029933 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:08.030081 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.030914 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.032078 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.032844 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.032920 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033005 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.033133 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.033175 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260811 10:16:08.033510 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:45503 (local address 127.0.0.1:49620)
0811 10:16:08.029339 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.029440 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.029511 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.029810 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.029813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.029815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:08.029898 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.029900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.031255 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.031256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.032354 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.032357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.033250 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.033251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.033316 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033320 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.033374 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.033416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260811 10:16:08.033814 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.8.238.2:45503)
0811 10:16:08.029496 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.029501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.029513 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:08.029635 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.029638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.029655 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.029708 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:08.030004 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.031155 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.032450 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.033068 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.033132 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033215 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.033672 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.033720 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260811 10:16:08.034065 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.8.238.3:35953)
0811 10:16:08.029692 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.029697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.030158 (+ 461us) server_negotiation.cc:407] Connection header received
0811 10:16:08.030198 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.030200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.030215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.030304 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:08.030455 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.031515 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.032619 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.033333 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.033714 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033806 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.033860 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.033989 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260811 10:16:08.034144 6852 tablet_service.cc:1467] Processing CreateTablet for tablet b2025992bf25456aa28d11b8b80733cd (DEFAULT_TABLE table=test-table [id=34bf8a3c784044eb916a1e999801abe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:08.034346 7472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.029410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:35953 (local address 127.0.0.1:38216)
0811 10:16:08.030029 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.030040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.030088 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.030314 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.030316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.030318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:08.030357 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.030359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.031604 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.031605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.032524 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.032527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.033718 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.033719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.033794 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.033798 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.033850 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.033970 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":76,"threads_started":1}
I20260811 10:16:08.034397 7102 tablet_service.cc:1467] Processing CreateTablet for tablet b2025992bf25456aa28d11b8b80733cd (DEFAULT_TABLE table=test-table [id=34bf8a3c784044eb916a1e999801abe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:08.034560 6852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2025992bf25456aa28d11b8b80733cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:08.034726 7102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2025992bf25456aa28d11b8b80733cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:08.036355 7338 tablet_service.cc:1467] Processing CreateTablet for tablet b2025992bf25456aa28d11b8b80733cd (DEFAULT_TABLE table=test-table [id=34bf8a3c784044eb916a1e999801abe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:08.036677 7338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2025992bf25456aa28d11b8b80733cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:08.038165 7488 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Bootstrap starting.
I20260811 10:16:08.038905 7488 tablet_bootstrap.cc:654] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:08.039245 7488 log.cc:824] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:08.039625 7489 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Bootstrap starting.
I20260811 10:16:08.039852 7491 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Bootstrap starting.
I20260811 10:16:08.040081 7488 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: No bootstrap required, opened a new log
I20260811 10:16:08.040156 7488 ts_tablet_manager.cc:1397] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:08.040357 7489 tablet_bootstrap.cc:654] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:08.040676 7489 log.cc:824] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:08.040688 7491 tablet_bootstrap.cc:654] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:08.041061 7491 log.cc:824] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:08.041469 7489 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: No bootstrap required, opened a new log
I20260811 10:16:08.041548 7489 ts_tablet_manager.cc:1397] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:08.041832 7491 tablet_bootstrap.cc:492] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: No bootstrap required, opened a new log
I20260811 10:16:08.041905 7491 ts_tablet_manager.cc:1397] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:08.043200 7488 raft_consensus.cc:348] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.043378 7488 raft_consensus.cc:374] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:08.043421 7488 raft_consensus.cc:729] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e59251cd8d04b30b84a1f7f0dfe5f31, State: Initialized, Role: FOLLOWER
I20260811 10:16:08.043465 7489 raft_consensus.cc:348] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.043560 7488 consensus_queue.cc:260] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.043632 7489 raft_consensus.cc:374] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:08.043678 7489 raft_consensus.cc:729] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d612db398434d8187f89e7fade415a1, State: Initialized, Role: FOLLOWER
I20260811 10:16:08.043851 7488 ts_tablet_manager.cc:1428] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 10:16:08.043840 7489 consensus_queue.cc:260] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.043995 6968 heartbeater.cc:503] Master 127.8.238.62:39251 was elected leader, sending a full tablet report...
I20260811 10:16:08.044449 7489 ts_tablet_manager.cc:1428] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:08.044571 7216 heartbeater.cc:503] Master 127.8.238.62:39251 was elected leader, sending a full tablet report...
I20260811 10:16:08.044592 7491 raft_consensus.cc:348] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.044770 7491 raft_consensus.cc:374] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:08.044814 7491 raft_consensus.cc:729] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c2fdfb4d949e8b6b5c8f906bc3d26, State: Initialized, Role: FOLLOWER
I20260811 10:16:08.044947 7491 consensus_queue.cc:260] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.045229 7491 ts_tablet_manager.cc:1428] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:16:08.045284 7461 heartbeater.cc:503] Master 127.8.238.62:39251 was elected leader, sending a full tablet report...
W20260811 10:16:08.126734 7217 tablet.cc:2369] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:16:08.248064 6969 tablet.cc:2369] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:16:08.261022 7462 tablet.cc:2369] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:08.262347 7495 raft_consensus.cc:482] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:08.262456 7495 raft_consensus.cc:504] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.262804 7495 leader_election.cc:288] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d612db398434d8187f89e7fade415a1 (127.8.238.2:45503), 842c2fdfb4d949e8b6b5c8f906bc3d26 (127.8.238.3:35953)
I20260811 10:16:08.265844 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.263256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:55869 (local address 127.8.238.3:35953)
0811 10:16:08.263314 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.263316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.263328 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:08.263395 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.263396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.263405 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.263458 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:08.263618 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.264162 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265109 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.265484 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265576 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.265586 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.265734 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.265770 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:16:08.265858 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.263089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:37963 (local address 127.8.238.2:45503)
0811 10:16:08.263150 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.263152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.263187 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:16:08.263263 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.263265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.263278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.263317 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:08.263580 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.264127 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265120 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.265489 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265568 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.265579 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.265748 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.265791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 10:16:08.265877 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.263174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:35953 (local address 127.8.238.1:55869)
0811 10:16:08.263263 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.263284 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.263340 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.263472 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.263473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.263475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:08.263542 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.263543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.264248 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.264249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.265019 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.265021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.265582 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.265660 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.265664 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.265728 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.265770 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":3,"thread_start_us":40,"threads_started":1}
I20260811 10:16:08.265877 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.262986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:45503 (local address 127.8.238.1:37963)
0811 10:16:08.263114 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.263127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:08.263174 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:08.263401 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:08.263402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:08.263405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:08.263498 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.263501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.264209 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.264210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.265009 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:08.265011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.265580 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.265582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:08.265670 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.265674 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:08.265740 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 10:16:08.265781 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":25,"thread_start_us":75,"threads_started":1}
I20260811 10:16:08.266323 7372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2025992bf25456aa28d11b8b80733cd" candidate_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" is_pre_election: true
I20260811 10:16:08.266510 7372 raft_consensus.cc:2454] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e59251cd8d04b30b84a1f7f0dfe5f31 in term 0.
I20260811 10:16:08.266489 7136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2025992bf25456aa28d11b8b80733cd" candidate_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d612db398434d8187f89e7fade415a1" is_pre_election: true
I20260811 10:16:08.266664 7136 raft_consensus.cc:2454] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e59251cd8d04b30b84a1f7f0dfe5f31 in term 0.
I20260811 10:16:08.266781 6764 leader_election.cc:302] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e59251cd8d04b30b84a1f7f0dfe5f31, 842c2fdfb4d949e8b6b5c8f906bc3d26; no voters:
I20260811 10:16:08.266937 7495 raft_consensus.cc:2781] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:08.266999 7495 raft_consensus.cc:482] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:08.267030 7495 raft_consensus.cc:3037] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:08.267908 7495 raft_consensus.cc:504] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.268054 7495 leader_election.cc:288] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [CANDIDATE]: Term 1 election: Requested vote from peers 6d612db398434d8187f89e7fade415a1 (127.8.238.2:45503), 842c2fdfb4d949e8b6b5c8f906bc3d26 (127.8.238.3:35953)
I20260811 10:16:08.268224 7136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2025992bf25456aa28d11b8b80733cd" candidate_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d612db398434d8187f89e7fade415a1"
I20260811 10:16:08.268235 7372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2025992bf25456aa28d11b8b80733cd" candidate_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26"
I20260811 10:16:08.268307 7136 raft_consensus.cc:3037] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:08.268344 7372 raft_consensus.cc:3037] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:08.269150 7136 raft_consensus.cc:2454] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e59251cd8d04b30b84a1f7f0dfe5f31 in term 1.
I20260811 10:16:08.269261 7372 raft_consensus.cc:2454] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e59251cd8d04b30b84a1f7f0dfe5f31 in term 1.
I20260811 10:16:08.269321 6764 leader_election.cc:302] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e59251cd8d04b30b84a1f7f0dfe5f31, 6d612db398434d8187f89e7fade415a1; no voters:
I20260811 10:16:08.269436 7495 raft_consensus.cc:2781] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:08.269630 7495 raft_consensus.cc:686] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [term 1 LEADER]: Becoming Leader. State: Replica: 1e59251cd8d04b30b84a1f7f0dfe5f31, State: Running, Role: LEADER
I20260811 10:16:08.269757 7495 consensus_queue.cc:237] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } }
I20260811 10:16:08.270633 6618 catalog_manager.cc:5495] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e59251cd8d04b30b84a1f7f0dfe5f31 (127.8.238.1). New cstate: current_term: 1 leader_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e59251cd8d04b30b84a1f7f0dfe5f31" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 34541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 } health_report { overall_health: UNKNOWN } } }
I20260811 10:16:08.318240 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.316028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.8.238.1:34541)
0811 10:16:08.316094 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:08.316096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:08.316106 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:08.316159 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:08.316159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:08.316182 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:08.316244 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:08.316372 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.316861 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.317585 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:08.317771 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:08.317842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:08.317962 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:08.318065 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:08.318128 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:08.318160 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 10:16:08.318177 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:16:08.320253 7136 raft_consensus.cc:1261] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 1 FOLLOWER]: Refusing update from remote peer 1e59251cd8d04b30b84a1f7f0dfe5f31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:08.320277 7372 raft_consensus.cc:1261] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 1 FOLLOWER]: Refusing update from remote peer 1e59251cd8d04b30b84a1f7f0dfe5f31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:08.320645 7495 consensus_queue.cc:1035] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d612db398434d8187f89e7fade415a1" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:08.320797 7505 consensus_queue.cc:1035] T b2025992bf25456aa28d11b8b80733cd P 1e59251cd8d04b30b84a1f7f0dfe5f31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "842c2fdfb4d949e8b6b5c8f906bc3d26" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:08.323401 7511 mvcc.cc:205] Tried to move back new op lower bound from 7317272036636823552 to 7317272036433686528. Current Snapshot: MvccSnapshot[applied={T|T < 7317272036636823552}]
I20260811 10:16:08.323701 7513 mvcc.cc:205] Tried to move back new op lower bound from 7317272036636823552 to 7317272036433686528. Current Snapshot: MvccSnapshot[applied={T|T < 7317272036636823552}]
I20260811 10:16:08.326110 6619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:08.326211 6619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:08.327044 6619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91d78072d16d4ecaa607d2fc665877a7: Sending DeleteTablet for 3 replicas of tablet b2025992bf25456aa28d11b8b80733cd
I20260811 10:16:08.327405 7102 tablet_service.cc:1514] Processing DeleteTablet for tablet b2025992bf25456aa28d11b8b80733cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:08 UTC) from {username='slave'} at 127.0.0.1:49620
I20260811 10:16:08.327505 7338 tablet_service.cc:1514] Processing DeleteTablet for tablet b2025992bf25456aa28d11b8b80733cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:08 UTC) from {username='slave'} at 127.0.0.1:38216
I20260811 10:16:08.327486 6852 tablet_service.cc:1514] Processing DeleteTablet for tablet b2025992bf25456aa28d11b8b80733cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:08 UTC) from {username='slave'} at 127.0.0.1:46204
I20260811 10:16:08.327780 7537 tablet_replica.cc:330] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: stopping tablet replica
I20260811 10:16:08.327890 7537 raft_consensus.cc:2229] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:08.328003 7537 raft_consensus.cc:2258] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:08.328043 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 6734
I20260811 10:16:08.327988 7534 tablet_replica.cc:330] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: stopping tablet replica
I20260811 10:16:08.341008 7534 raft_consensus.cc:2229] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:08.341153 7534 raft_consensus.cc:2258] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:08.341316 7537 ts_tablet_manager.cc:1905] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:08.341681 7534 ts_tablet_manager.cc:1905] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:08.343562 7534 ts_tablet_manager.cc:1918] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:08.343658 7534 log.cc:1196] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/b2025992bf25456aa28d11b8b80733cd
I20260811 10:16:08.343698 7537 ts_tablet_manager.cc:1918] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:08.343801 7537 log.cc:1196] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/b2025992bf25456aa28d11b8b80733cd
I20260811 10:16:08.343997 7534 ts_tablet_manager.cc:1939] T b2025992bf25456aa28d11b8b80733cd P 6d612db398434d8187f89e7fade415a1: Deleting consensus metadata
I20260811 10:16:08.344060 7537 ts_tablet_manager.cc:1939] T b2025992bf25456aa28d11b8b80733cd P 842c2fdfb4d949e8b6b5c8f906bc3d26: Deleting consensus metadata
I20260811 10:16:08.344506 6581 catalog_manager.cc:4841] TS 842c2fdfb4d949e8b6b5c8f906bc3d26 (127.8.238.3:35953): tablet b2025992bf25456aa28d11b8b80733cd (table test-table [id=34bf8a3c784044eb916a1e999801abe3]) successfully deleted
I20260811 10:16:08.344655 6581 catalog_manager.cc:4841] TS 6d612db398434d8187f89e7fade415a1 (127.8.238.2:45503): tablet b2025992bf25456aa28d11b8b80733cd (table test-table [id=34bf8a3c784044eb916a1e999801abe3]) successfully deleted
W20260811 10:16:08.346480 6583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:34541 (error 108)
I20260811 10:16:08.346676 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 6971
I20260811 10:16:08.359393 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:08.359198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:34541 (local address 127.0.0.1:46220)
0811 10:16:08.359278 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:08.359349 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 10:16:08.359572 6583 catalog_manager.cc:4584] TS 1e59251cd8d04b30b84a1f7f0dfe5f31 (127.8.238.1:34541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2025992bf25456aa28d11b8b80733cd: Network error: Client connection negotiation failed: client connection to 127.8.238.1:34541: connect: Connection refused (error 111)
I20260811 10:16:08.363708 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 7224
I20260811 10:16:08.373445 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 6559
2026-08-11T10:16:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (939 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:08 dist-test-slave-7f01 krb5kdc[7544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:08 dist-test-slave-7f01 krb5kdc[7544](info): set up 2 sockets
Aug 11 10:16:08 dist-test-slave-7f01 krb5kdc[7544](info): commencing operation
krb5kdc: starting...
W20260811 10:16:09.677878 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:09 dist-test-slave-7f01 krb5kdc[7544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:09.828667 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40039
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:40039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:09.928112 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:09.928365 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:09.928390 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:09.929518 7572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:09.929556 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:09.929570 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:09.929584 7572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:09.929596 7572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:09.931054 7572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:40039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:40039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:09.931260 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:09.931452 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:09.936524 7579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:09.936546 7580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:09.936600 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:09.936874 7572 server_base.cc:1034] running on GCE node
I20260811 10:16:09.937217 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:09.937597 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:09.938796 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786443369938761 us; error 54 us; skew 500 ppm
Aug 11 10:16:09 dist-test-slave-7f01 krb5kdc[7544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:09.942251 7572 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:09.942986 7572 webserver.cc:460] Webserver started at http://127.8.238.62:35977/ using document root <none> and password file <none>
I20260811 10:16:09.943315 7572 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:09.943395 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:09.943583 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:09.945056 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "3ab43346693e449094601338e6ea4027"
format_stamp: "Formatted at 2026-08-11 10:16:09 on dist-test-slave-7f01"
server_key: "5a29d15ecd92c7b7d1cddf9e490e0d52"
server_key_iv: "0c18f51c77bda22ea2ada492ab426620"
server_key_version: "encryptionkey@0"
I20260811 10:16:09.945535 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "3ab43346693e449094601338e6ea4027"
format_stamp: "Formatted at 2026-08-11 10:16:09 on dist-test-slave-7f01"
server_key: "5a29d15ecd92c7b7d1cddf9e490e0d52"
server_key_iv: "0c18f51c77bda22ea2ada492ab426620"
server_key_version: "encryptionkey@0"
I20260811 10:16:09.947693 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:09.949074 7597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:09.949458 7572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:09.949618 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "3ab43346693e449094601338e6ea4027"
format_stamp: "Formatted at 2026-08-11 10:16:09 on dist-test-slave-7f01"
server_key: "5a29d15ecd92c7b7d1cddf9e490e0d52"
server_key_iv: "0c18f51c77bda22ea2ada492ab426620"
server_key_version: "encryptionkey@0"
I20260811 10:16:09.949728 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:09.968538 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:09.971704 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:09.971982 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:09.976627 7572 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:40039
I20260811 10:16:09.976668 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:40039 every 8 connection(s)
I20260811 10:16:09.977074 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:09.978223 7695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:09.981582 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: Bootstrap starting.
I20260811 10:16:09.982452 7695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:09.982854 7695 log.cc:824] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:09.984055 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: No bootstrap required, opened a new log
I20260811 10:16:09.986562 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 7572
I20260811 10:16:09.986531 7695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } }
I20260811 10:16:09.986742 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:09.986802 7695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:09.986842 7695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab43346693e449094601338e6ea4027, State: Initialized, Role: FOLLOWER
I20260811 10:16:09.987025 9144 external_mini_cluster.cc:1428] Setting key 7003fb74e7b8ed9dfbe7f5b463242778
I20260811 10:16:09.987028 7695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } }
I20260811 10:16:09.987134 7695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:09.987187 7695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:09.987250 7695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:09.988941 7695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } }
I20260811 10:16:09.989094 7695 leader_election.cc:302] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ab43346693e449094601338e6ea4027; no voters:
I20260811 10:16:09.989311 7695 leader_election.cc:288] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:09.989423 7703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:09.989657 7695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:09.989754 7703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab43346693e449094601338e6ea4027, State: Running, Role: LEADER
I20260811 10:16:09.990065 7703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } }
I20260811 10:16:09.990890 7704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ab43346693e449094601338e6ea4027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } } }
I20260811 10:16:09.991031 7704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:09.991458 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ab43346693e449094601338e6ea4027. Latest consensus state: current_term: 1 leader_uuid: "3ab43346693e449094601338e6ea4027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab43346693e449094601338e6ea4027" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 40039 } } }
I20260811 10:16:09.991606 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027 [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:09.991863 7728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:09.992151 7728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:09.992312 7724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:09.992488 7724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:09.994431 7724 catalog_manager.cc:1347] Generated new cluster ID: 86ee3553f45a487980a3cacf8a7de0ba
I20260811 10:16:09.994491 7724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:10.004654 7724 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:10.005714 7724 catalog_manager.cc:1504] Loading token signing keys...
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:10.013135 7724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: Generated new TSK 0
I20260811 10:16:10.013353 7724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:10.018718 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:09.987864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.8.238.62:40039)
0811 10:16:09.988273 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:09.988289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:09.988344 (+ 55us) server_negotiation.cc:407] Connection header received
0811 10:16:09.988534 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:09.988542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:09.988614 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:09.988797 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:09.989907 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:09.991682 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:09.993418 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:09.994093 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:09.994217 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.013721 (+ 19504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.013728 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.013728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:10.013743 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.016263 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.016818 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.016822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.016822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.016945 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.017146 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.017147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.017147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.017515 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.018178 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.018259 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.018446 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":190,"threads_started":1}
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:16:10.052042 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:10.149577 7739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:10.149832 7739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:10.149860 7739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:10.151108 7739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:10.151154 7739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:10.151181 7739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:10.152900 7739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:10.153177 7739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:10.153448 7739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:10.159546 7754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.159732 7758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.159513 7752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.160164 7739 server_base.cc:1034] running on GCE node
I20260811 10:16:10.160524 7739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:10.160897 7739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:10.162176 7739 hybrid_clock.cc:648] HybridClock initialized: now 1786443370162163 us; error 52 us; skew 500 ppm
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:10.165539 7739 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:16:10.166157 7739 webserver.cc:460] Webserver started at http://127.8.238.1:40135/ using document root <none> and password file <none>
I20260811 10:16:10.166446 7739 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:10.166520 7739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:10.166700 7739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:10.168125 7739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "6d242cd640df40f89f5fb0f3c82c6ac4"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "d8e26d183416f7d9656f1644e9688f70"
server_key_iv: "d4bcb248f6ee7668dff3ed1622b70a78"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.168543 7739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "6d242cd640df40f89f5fb0f3c82c6ac4"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "d8e26d183416f7d9656f1644e9688f70"
server_key_iv: "d4bcb248f6ee7668dff3ed1622b70a78"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.170295 7739 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:10.171514 7772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.171900 7739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:10.172019 7739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "6d242cd640df40f89f5fb0f3c82c6ac4"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "d8e26d183416f7d9656f1644e9688f70"
server_key_iv: "d4bcb248f6ee7668dff3ed1622b70a78"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.172122 7739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:10.185316 7739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:10.188743 7739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:10.188911 7739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:10.189222 7739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:10.189702 7739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:10.189810 7739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.189834 7739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:10.189877 7739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.196790 7739 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:36987
I20260811 10:16:10.196893 7984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:36987 every 8 connection(s)
I20260811 10:16:10.197227 7739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:16:10.199635 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 7739
I20260811 10:16:10.199782 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:10.200098 9144 external_mini_cluster.cc:1428] Setting key f2c847321e3cddf34f453c6ec342a55a
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:10.210866 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.198095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:51987 (local address 127.8.238.62:40039)
0811 10:16:10.198232 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.198234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.198431 (+ 197us) server_negotiation.cc:407] Connection header received
0811 10:16:10.199035 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.199036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.199049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.199123 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:10.200403 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.201015 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.202698 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.202993 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.203061 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.206376 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.206379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.206379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:10.206390 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.208811 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.209427 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.209429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.209430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.209527 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.209837 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.209838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.209839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.210040 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.210135 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.210573 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.210714 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:16:10.211033 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.198046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40039 (local address 127.8.238.1:51987)
0811 10:16:10.198275 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.198333 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.198924 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.199196 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.199200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.199999 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:10.200270 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.200276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.201232 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.201235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.202494 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.202499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.203116 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.203118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.203317 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.204146 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:10.204159 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:10.206231 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:10.208987 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.208998 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.209001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.209312 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.209622 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.209623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.209623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.209750 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.210180 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:10.210193 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:10.210392 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.210516 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.210753 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260811 10:16:10.212013 7989 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40039
I20260811 10:16:10.212162 7989 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:10.212409 7989 heartbeater.cc:511] Master 127.8.238.62:40039 requested a full tablet report, sending...
I20260811 10:16:10.213174 7636 ts_manager.cc:194] Registered new tserver with Master: 6d242cd640df40f89f5fb0f3c82c6ac4 (127.8.238.1:36987)
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
I20260811 10:16:10.214021 7636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:51987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:16:10.240828 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:10.336804 7995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:10.337265 7995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:10.337329 7995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:10.339347 7995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:10.339463 7995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:10.339529 7995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:10.342664 7995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:10.343107 7995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:10.343454 7995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:10.350229 8017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.350242 8010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.350526 7995 server_base.cc:1034] running on GCE node
W20260811 10:16:10.350265 8009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.350952 7995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:10.351316 7995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:10.352573 7995 hybrid_clock.cc:648] HybridClock initialized: now 1786443370352522 us; error 92 us; skew 500 ppm
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:10.356199 7995 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:16:10.356884 7995 webserver.cc:460] Webserver started at http://127.8.238.2:39857/ using document root <none> and password file <none>
I20260811 10:16:10.357183 7995 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:10.357266 7995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:10.357475 7995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:10.358737 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "a15a6a5cafe549dab6db3a67694a49bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "e2faa5327fcc4658dfef66c8e4d4de82"
server_key_iv: "1d6c514af3b7a2aa1c8615bc9b994a1e"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.359097 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "a15a6a5cafe549dab6db3a67694a49bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "e2faa5327fcc4658dfef66c8e4d4de82"
server_key_iv: "1d6c514af3b7a2aa1c8615bc9b994a1e"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.360847 7995 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 10:16:10.361963 8029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.362264 7995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:10.362382 7995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "a15a6a5cafe549dab6db3a67694a49bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "e2faa5327fcc4658dfef66c8e4d4de82"
server_key_iv: "1d6c514af3b7a2aa1c8615bc9b994a1e"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.362461 7995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:10.382966 7995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:10.385555 7995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:10.385685 7995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:10.385905 7995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:10.386274 7995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:10.386312 7995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.386335 7995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:10.386350 7995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.394191 7995 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:35681
I20260811 10:16:10.394382 8249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:35681 every 8 connection(s)
I20260811 10:16:10.394651 7995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:16:10.398422 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 7995
I20260811 10:16:10.398551 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:10.398732 9144 external_mini_cluster.cc:1428] Setting key c8d08f1855e66c72f5c54ce2cefef4a8
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:10.407083 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.395883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:49065 (local address 127.8.238.62:40039)
0811 10:16:10.395986 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.395988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.396304 (+ 316us) server_negotiation.cc:407] Connection header received
0811 10:16:10.396914 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.396916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.396929 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.396993 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:10.398066 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.398890 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.400109 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.400334 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.400424 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.403078 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.403081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.403081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:10.403090 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.405153 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.405850 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.405851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.405851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.405915 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.406291 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.406292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.406292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.406433 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.406503 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.406916 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.406993 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 10:16:10.407548 8257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.395841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40039 (local address 127.8.238.2:49065)
0811 10:16:10.396133 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.396177 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.396794 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.397073 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.397078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.397705 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:10.397961 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.397966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.398980 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.398982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.400019 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.400022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.400438 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.400439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.400583 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.401201 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:10.401211 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:10.402953 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:10.405254 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.405266 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.405269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.405749 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.406001 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.406002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.406003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.406190 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.406517 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:10.406527 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:10.406740 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.406901 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.407130 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":170,"threads_started":1}
I20260811 10:16:10.408200 8250 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40039
I20260811 10:16:10.408334 8250 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:10.408581 8250 heartbeater.cc:511] Master 127.8.238.62:40039 requested a full tablet report, sending...
I20260811 10:16:10.409242 7636 ts_manager.cc:194] Registered new tserver with Master: a15a6a5cafe549dab6db3a67694a49bb (127.8.238.2:35681)
I20260811 10:16:10.409660 7636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:49065
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:16:10.436806 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:10.530452 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:10.530691 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:10.530715 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:10.531973 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:10.532029 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:10.532058 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:10.533695 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:40039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:10.533981 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:10.534241 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:10.539858 8277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.539999 8266 server_base.cc:1034] running on GCE node
W20260811 10:16:10.539968 8287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.539887 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.540589 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:10.540990 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:10.542223 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786443370542189 us; error 47 us; skew 500 ppm
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:10.545435 8266 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:16:10.545981 8266 webserver.cc:460] Webserver started at http://127.8.238.3:44727/ using document root <none> and password file <none>
I20260811 10:16:10.546242 8266 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:10.546317 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:10.546476 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:10.547608 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "309361440bc14b12aae5d837573443bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "0a3083ca62ab3d533866abd5d6bf14bb"
server_key_iv: "552836761d4cec93b603cddf27d73f61"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.548054 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "309361440bc14b12aae5d837573443bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "0a3083ca62ab3d533866abd5d6bf14bb"
server_key_iv: "552836761d4cec93b603cddf27d73f61"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.549911 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:10.551153 8301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.551451 8266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:10.551591 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "309361440bc14b12aae5d837573443bb"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "0a3083ca62ab3d533866abd5d6bf14bb"
server_key_iv: "552836761d4cec93b603cddf27d73f61"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.551683 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:10.563316 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:10.566440 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:10.566715 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:10.567092 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:10.567520 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:10.567677 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.567843 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:10.567890 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.577957 8266 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:35613
I20260811 10:16:10.578027 8521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:35613 every 8 connection(s)
I20260811 10:16:10.578451 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
Aug 11 10:16:10 dist-test-slave-7f01 krb5kdc[7544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:10.587805 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 8266
I20260811 10:16:10.587934 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:10.588145 9144 external_mini_cluster.cc:1428] Setting key 201aa9e048811779124c81fffc953e91
I20260811 10:16:10.591259 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.579881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:58307 (local address 127.8.238.62:40039)
0811 10:16:10.579978 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.579981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.580206 (+ 225us) server_negotiation.cc:407] Connection header received
0811 10:16:10.581040 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.581041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.581055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.581134 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:10.582243 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.582910 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.584128 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.584337 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.584404 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.587410 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.587412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.587412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:10.587419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.589223 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.589698 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.589699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.589700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.589777 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.590218 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.590221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.590221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.590483 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.590605 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.591051 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.591165 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 10:16:10.591538 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.579669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:40039 (local address 127.8.238.3:58307)
0811 10:16:10.579994 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.580058 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.580908 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.581220 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.581224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.581894 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:10.582107 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.582112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.583012 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.583013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.584034 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.584037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.584427 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.584427 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.584540 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.585169 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:10.585181 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:10.587167 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:10.589316 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.589324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.589326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.589598 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.589915 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:10.589916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:10.589916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:10.590058 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:10.590601 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:10.590613 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:10.590801 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.590923 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.591226 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":167,"threads_started":1}
I20260811 10:16:10.592398 8523 heartbeater.cc:347] Connected to a master server at 127.8.238.62:40039
I20260811 10:16:10.592556 8523 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:10.592799 8523 heartbeater.cc:511] Master 127.8.238.62:40039 requested a full tablet report, sending...
I20260811 10:16:10.593379 7636 ts_manager.cc:194] Registered new tserver with Master: 309361440bc14b12aae5d837573443bb (127.8.238.3:35613)
I20260811 10:16:10.593819 7636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:58307
I20260811 10:16:10.600417 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:10.609630 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.602064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.8.238.62:40039)
0811 10:16:10.602179 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.602182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.602292 (+ 110us) server_negotiation.cc:407] Connection header received
0811 10:16:10.602409 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.602410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.602420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.602494 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:10.603092 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.603601 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.604416 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.604609 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.604686 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.606148 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.606150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.606150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:10.606158 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.607997 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.608410 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.608411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.608412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.608475 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:10.608703 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:10.608704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:10.608704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:10.608849 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.608980 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.609443 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.609512 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 10:16:10.612296 7636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:10.614471 7636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:10.623585 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:35681 (local address 127.0.0.1:55064)
0811 10:16:10.618803 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.618882 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.618959 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.619514 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.619515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.619519 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:10.619653 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.619657 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.620867 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.620868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.622199 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.622203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.623130 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.623131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.623248 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.623268 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.623385 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.623470 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":82,"thread_start_us":155,"threads_started":1}
I20260811 10:16:10.623612 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55064 (local address 127.8.238.2:35681)
0811 10:16:10.619282 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.619287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.619300 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:16:10.619353 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.619355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.619378 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.619795 (+ 417us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:10.619885 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.620706 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.622312 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.623010 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.623103 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.623177 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.623427 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.623485 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":171,"threads_started":1}
I20260811 10:16:10.624429 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36987 (local address 127.0.0.1:54170)
0811 10:16:10.618883 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.618903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.618997 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.619624 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.619625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.619627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:10.619667 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.619669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.621199 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.621200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.622765 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.622769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.624035 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.624037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.624149 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.624156 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.624230 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.624302 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":96,"thread_start_us":145,"threads_started":1}
I20260811 10:16:10.624773 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:35613 (local address 127.0.0.1:43914)
0811 10:16:10.619124 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.619136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.619216 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.619495 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.619499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.619501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:10.619642 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.619647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.621266 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.621267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.622789 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.622792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.624419 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.624420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.624610 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.624616 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.624674 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.624722 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":100,"thread_start_us":109,"threads_started":1}
I20260811 10:16:10.624956 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43914 (local address 127.8.238.3:35613)
0811 10:16:10.619120 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.619127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.619238 (+ 111us) server_negotiation.cc:407] Connection header received
0811 10:16:10.619311 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.619314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.619336 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.619456 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:10.619813 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.621140 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.622906 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.624301 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.624391 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.624463 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.624697 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.624747 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":141,"threads_started":1}
I20260811 10:16:10.624873 8131 tablet_service.cc:1467] Processing CreateTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 (DEFAULT_TABLE table=test-table [id=1f2b16073e0247fdafd5261636c7bdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:10.625337 8131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d57390cd75c4b4eb5bed48edc7d6789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:10.625886 8396 tablet_service.cc:1467] Processing CreateTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 (DEFAULT_TABLE table=test-table [id=1f2b16073e0247fdafd5261636c7bdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:10.626257 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d57390cd75c4b4eb5bed48edc7d6789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:10.628047 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.618523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.8.238.1:36987)
0811 10:16:10.619407 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.619413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.619452 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:16:10.619499 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.619501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.619520 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.619626 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:10.619731 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.621054 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.622898 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.623824 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.623915 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.623983 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.627890 (+ 3907us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.627960 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":147,"threads_started":1}
I20260811 10:16:10.628770 7863 tablet_service.cc:1467] Processing CreateTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 (DEFAULT_TABLE table=test-table [id=1f2b16073e0247fdafd5261636c7bdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:10.629118 7863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d57390cd75c4b4eb5bed48edc7d6789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:10.630048 8558 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Bootstrap starting.
I20260811 10:16:10.630193 8556 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Bootstrap starting.
I20260811 10:16:10.630851 8558 tablet_bootstrap.cc:654] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:10.630959 8556 tablet_bootstrap.cc:654] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:10.631443 8558 log.cc:824] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:10.631587 8556 log.cc:824] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:10.632299 8558 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: No bootstrap required, opened a new log
I20260811 10:16:10.632366 8558 ts_tablet_manager.cc:1397] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:10.632797 8561 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Bootstrap starting.
I20260811 10:16:10.633245 8556 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: No bootstrap required, opened a new log
I20260811 10:16:10.633339 8556 ts_tablet_manager.cc:1397] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:16:10.633508 8561 tablet_bootstrap.cc:654] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:10.634164 8561 log.cc:824] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:10.635063 8561 tablet_bootstrap.cc:492] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: No bootstrap required, opened a new log
I20260811 10:16:10.635159 8561 ts_tablet_manager.cc:1397] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:10.635059 8556 raft_consensus.cc:348] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.635269 8556 raft_consensus.cc:374] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:10.635164 8558 raft_consensus.cc:348] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.635321 8556 raft_consensus.cc:729] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a15a6a5cafe549dab6db3a67694a49bb, State: Initialized, Role: FOLLOWER
I20260811 10:16:10.635332 8558 raft_consensus.cc:374] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:10.635366 8558 raft_consensus.cc:729] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309361440bc14b12aae5d837573443bb, State: Initialized, Role: FOLLOWER
I20260811 10:16:10.635444 8556 consensus_queue.cc:260] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.635536 8558 consensus_queue.cc:260] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.635790 8556 ts_tablet_manager.cc:1428] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:10.635881 8558 ts_tablet_manager.cc:1428] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 10:16:10.635926 8250 heartbeater.cc:503] Master 127.8.238.62:40039 was elected leader, sending a full tablet report...
I20260811 10:16:10.635936 8523 heartbeater.cc:503] Master 127.8.238.62:40039 was elected leader, sending a full tablet report...
I20260811 10:16:10.637734 8561 raft_consensus.cc:348] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.637876 8561 raft_consensus.cc:374] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:10.637900 8561 raft_consensus.cc:729] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d242cd640df40f89f5fb0f3c82c6ac4, State: Initialized, Role: FOLLOWER
I20260811 10:16:10.638021 8561 consensus_queue.cc:260] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.638310 8561 ts_tablet_manager.cc:1428] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:10.638411 7989 heartbeater.cc:503] Master 127.8.238.62:40039 was elected leader, sending a full tablet report...
W20260811 10:16:10.645473 8256 tablet.cc:2369] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:10.669186 8564 raft_consensus.cc:482] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:10.669363 8564 raft_consensus.cc:504] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.669843 8564 leader_election.cc:288] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d242cd640df40f89f5fb0f3c82c6ac4 (127.8.238.1:36987), a15a6a5cafe549dab6db3a67694a49bb (127.8.238.2:35681)
I20260811 10:16:10.673575 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.670129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:38169 (local address 127.8.238.1:36987)
0811 10:16:10.670215 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.670218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.670250 (+ 32us) server_negotiation.cc:407] Connection header received
0811 10:16:10.670287 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.670288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.670304 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.670388 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:10.670579 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.671261 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.672725 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.673132 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.673199 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.673238 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.673452 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.673496 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 10:16:10.673594 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.670221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:35395 (local address 127.8.238.2:35681)
0811 10:16:10.670276 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.670278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.670388 (+ 110us) server_negotiation.cc:407] Connection header received
0811 10:16:10.670465 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.670467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.670482 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.670566 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:10.670749 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.671463 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.672654 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.673140 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.673255 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.673270 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.673452 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.673501 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:16:10.673645 8527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.670037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36987 (local address 127.8.238.3:38169)
0811 10:16:10.670126 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.670139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.670188 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.670396 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.670397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.670399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:10.670494 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.670496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.671479 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.671481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.672582 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.672591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.673244 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.673245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.673346 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.673350 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.673427 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.673496 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 10:16:10.673630 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.670051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:35681 (local address 127.8.238.3:35395)
0811 10:16:10.670302 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.670312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:10.670397 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:10.670572 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:10.670573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:10.670589 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:10.670667 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.670669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.671581 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.671582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.672511 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:10.672514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.673283 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.673285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:10.673345 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.673350 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:10.673425 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:16:10.673496 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":146,"threads_started":1}
I20260811 10:16:10.674197 7908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d57390cd75c4b4eb5bed48edc7d6789" candidate_uuid: "309361440bc14b12aae5d837573443bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" is_pre_election: true
I20260811 10:16:10.674391 7908 raft_consensus.cc:2454] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 309361440bc14b12aae5d837573443bb in term 0.
I20260811 10:16:10.674552 8160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d57390cd75c4b4eb5bed48edc7d6789" candidate_uuid: "309361440bc14b12aae5d837573443bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15a6a5cafe549dab6db3a67694a49bb" is_pre_election: true
I20260811 10:16:10.674801 8160 raft_consensus.cc:2454] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 309361440bc14b12aae5d837573443bb in term 0.
I20260811 10:16:10.674769 8303 leader_election.cc:302] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309361440bc14b12aae5d837573443bb, 6d242cd640df40f89f5fb0f3c82c6ac4; no voters:
I20260811 10:16:10.674970 8564 raft_consensus.cc:2781] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:10.675074 8564 raft_consensus.cc:482] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:10.675099 8564 raft_consensus.cc:3037] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:10.676309 8564 raft_consensus.cc:504] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.676513 8564 leader_election.cc:288] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [CANDIDATE]: Term 1 election: Requested vote from peers 6d242cd640df40f89f5fb0f3c82c6ac4 (127.8.238.1:36987), a15a6a5cafe549dab6db3a67694a49bb (127.8.238.2:35681)
I20260811 10:16:10.676744 7908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d57390cd75c4b4eb5bed48edc7d6789" candidate_uuid: "309361440bc14b12aae5d837573443bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4"
I20260811 10:16:10.676846 7908 raft_consensus.cc:3037] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:10.677745 7908 raft_consensus.cc:2454] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 309361440bc14b12aae5d837573443bb in term 1.
I20260811 10:16:10.677983 8303 leader_election.cc:302] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309361440bc14b12aae5d837573443bb, 6d242cd640df40f89f5fb0f3c82c6ac4; no voters:
I20260811 10:16:10.677944 8160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d57390cd75c4b4eb5bed48edc7d6789" candidate_uuid: "309361440bc14b12aae5d837573443bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a15a6a5cafe549dab6db3a67694a49bb"
I20260811 10:16:10.678094 8564 raft_consensus.cc:2781] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:10.678081 8160 raft_consensus.cc:3037] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:10.678349 8564 raft_consensus.cc:686] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 1 LEADER]: Becoming Leader. State: Replica: 309361440bc14b12aae5d837573443bb, State: Running, Role: LEADER
I20260811 10:16:10.678479 8564 consensus_queue.cc:237] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } }
I20260811 10:16:10.679652 8160 raft_consensus.cc:2454] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 309361440bc14b12aae5d837573443bb in term 1.
I20260811 10:16:10.679942 7635 catalog_manager.cc:5495] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 309361440bc14b12aae5d837573443bb (127.8.238.3). New cstate: current_term: 1 leader_uuid: "309361440bc14b12aae5d837573443bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "309361440bc14b12aae5d837573443bb" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35613 } health_report { overall_health: HEALTHY } } }
W20260811 10:16:10.698179 7990 tablet.cc:2369] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:10.700861 8553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.698089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.8.238.3:35613)
0811 10:16:10.698171 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.698174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.698181 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:16:10.698229 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.698230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.698259 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.698373 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:10.698560 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.699093 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.699959 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.700164 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.700252 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.700430 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:10.700538 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:10.700658 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.700721 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.700754 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 10:16:10.703356 8160 raft_consensus.cc:1261] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 1 FOLLOWER]: Refusing update from remote peer 309361440bc14b12aae5d837573443bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:10.703538 7908 raft_consensus.cc:1261] T 5d57390cd75c4b4eb5bed48edc7d6789 P 6d242cd640df40f89f5fb0f3c82c6ac4 [term 1 FOLLOWER]: Refusing update from remote peer 309361440bc14b12aae5d837573443bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:10.703853 8567 consensus_queue.cc:1035] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a15a6a5cafe549dab6db3a67694a49bb" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 35681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:10.704036 8564 consensus_queue.cc:1035] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d242cd640df40f89f5fb0f3c82c6ac4" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:10.706774 8579 mvcc.cc:205] Tried to move back new op lower bound from 7317272046397198336 to 7317272046299910144. Current Snapshot: MvccSnapshot[applied={T|T < 7317272046397198336}]
I20260811 10:16:10.706818 8581 mvcc.cc:205] Tried to move back new op lower bound from 7317272046397198336 to 7317272046299910144. Current Snapshot: MvccSnapshot[applied={T|T < 7317272046397198336}]
I20260811 10:16:10.706945 8580 mvcc.cc:205] Tried to move back new op lower bound from 7317272046397198336 to 7317272046299910144. Current Snapshot: MvccSnapshot[applied={T|T < 7317272046397198336}]
I20260811 10:16:10.709967 7636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:10.710080 7636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:10.711030 7636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ab43346693e449094601338e6ea4027: Sending DeleteTablet for 3 replicas of tablet 5d57390cd75c4b4eb5bed48edc7d6789
I20260811 10:16:10.711426 8131 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:10 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:55064
I20260811 10:16:10.711447 7863 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:10 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:54170
I20260811 10:16:10.711483 8396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d57390cd75c4b4eb5bed48edc7d6789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:10 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:43914
I20260811 10:16:10.711696 8607 tablet_replica.cc:330] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: stopping tablet replica
I20260811 10:16:10.711838 8607 raft_consensus.cc:2229] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:16:10.711936 8598 tablet_replica.cc:330] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: stopping tablet replica
I20260811 10:16:10.711987 8607 raft_consensus.cc:2258] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:10.712031 8598 raft_consensus.cc:2229] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:10.712073 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 7739
I20260811 10:16:10.712123 8598 raft_consensus.cc:2258] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:10.716150 8598 ts_tablet_manager.cc:1905] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:10.717988 8598 ts_tablet_manager.cc:1918] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:10.718055 8598 log.cc:1196] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/5d57390cd75c4b4eb5bed48edc7d6789
I20260811 10:16:10.718308 8598 ts_tablet_manager.cc:1939] T 5d57390cd75c4b4eb5bed48edc7d6789 P a15a6a5cafe549dab6db3a67694a49bb: Deleting consensus metadata
I20260811 10:16:10.718777 7603 catalog_manager.cc:4841] TS a15a6a5cafe549dab6db3a67694a49bb (127.8.238.2:35681): tablet 5d57390cd75c4b4eb5bed48edc7d6789 (table test-table [id=1f2b16073e0247fdafd5261636c7bdf2]) successfully deleted
I20260811 10:16:10.725387 8607 ts_tablet_manager.cc:1905] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:10.727854 8607 ts_tablet_manager.cc:1918] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:10.727967 8607 log.cc:1196] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/5d57390cd75c4b4eb5bed48edc7d6789
I20260811 10:16:10.728307 8607 ts_tablet_manager.cc:1939] T 5d57390cd75c4b4eb5bed48edc7d6789 P 309361440bc14b12aae5d837573443bb: Deleting consensus metadata
I20260811 10:16:10.729101 7604 catalog_manager.cc:4841] TS 309361440bc14b12aae5d837573443bb (127.8.238.3:35613): tablet 5d57390cd75c4b4eb5bed48edc7d6789 (table test-table [id=1f2b16073e0247fdafd5261636c7bdf2]) successfully deleted
W20260811 10:16:10.736436 7604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:36987 (error 108)
I20260811 10:16:10.737169 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.736992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:36987 (local address 127.0.0.1:54178)
0811 10:16:10.737076 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:10.737127 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:36987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 10:16:10.737313 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 7995
W20260811 10:16:10.747148 7604 catalog_manager.cc:4584] TS 6d242cd640df40f89f5fb0f3c82c6ac4 (127.8.238.1:36987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d57390cd75c4b4eb5bed48edc7d6789: Network error: Client connection negotiation failed: client connection to 127.8.238.1:36987: connect: Connection refused (error 111)
I20260811 10:16:10.752759 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 8266
I20260811 10:16:10.770062 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 7572
2026-08-11T10:16:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2402 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T10:16:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:10Z Disabled control of system clock
I20260811 10:16:10.811625 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:37903
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:10.902870 8617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:10.903206 8617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:10.903265 8617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:10.905333 8617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:10.905386 8617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:10.905400 8617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:10.905416 8617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:10.905431 8617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:10.906893 8617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:37903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:10.907176 8617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:10.907521 8617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:10.912783 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.912809 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:10.912828 8631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:10.912964 8617 server_base.cc:1034] running on GCE node
I20260811 10:16:10.913295 8617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:10.913623 8617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:10.914829 8617 hybrid_clock.cc:648] HybridClock initialized: now 1786443370914774 us; error 72 us; skew 500 ppm
I20260811 10:16:10.916687 8617 webserver.cc:460] Webserver started at http://127.8.238.62:41319/ using document root <none> and password file <none>
I20260811 10:16:10.917043 8617 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:10.917124 8617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:10.917294 8617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:10.918619 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "e13ae747c248482cbcc2359fe3796268"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "73bdb52093c7558d1ed87fe825d1a064"
server_key_iv: "24346bacd2bb95f6fdaeebb3c911c560"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.919044 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "e13ae747c248482cbcc2359fe3796268"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "73bdb52093c7558d1ed87fe825d1a064"
server_key_iv: "24346bacd2bb95f6fdaeebb3c911c560"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.920912 8617 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260811 10:16:10.922061 8638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:10.922338 8617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:10.922490 8617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e13ae747c248482cbcc2359fe3796268"
format_stamp: "Formatted at 2026-08-11 10:16:10 on dist-test-slave-7f01"
server_key: "73bdb52093c7558d1ed87fe825d1a064"
server_key_iv: "24346bacd2bb95f6fdaeebb3c911c560"
server_key_version: "encryptionkey@0"
I20260811 10:16:10.922578 8617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:10.945658 8617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:10.945999 8617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:10.946120 8617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:10.950912 8617 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:37903
I20260811 10:16:10.951051 8732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:37903 every 8 connection(s)
I20260811 10:16:10.951365 8617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:10.952723 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:10.955798 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: Bootstrap starting.
I20260811 10:16:10.956676 8733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:10.957075 8733 log.cc:824] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:10.957948 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: No bootstrap required, opened a new log
I20260811 10:16:10.958376 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 8617
I20260811 10:16:10.958544 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:10.958775 9144 external_mini_cluster.cc:1428] Setting key 59979f0ab9ed7fa734f255c20ffb8a4e
I20260811 10:16:10.960768 8733 raft_consensus.cc:348] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } }
I20260811 10:16:10.961019 8733 raft_consensus.cc:374] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:10.961074 8733 raft_consensus.cc:729] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13ae747c248482cbcc2359fe3796268, State: Initialized, Role: FOLLOWER
I20260811 10:16:10.961364 8733 consensus_queue.cc:260] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } }
I20260811 10:16:10.961521 8733 raft_consensus.cc:388] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:10.961572 8733 raft_consensus.cc:482] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:10.961632 8733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:10.962386 8733 raft_consensus.cc:504] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } }
I20260811 10:16:10.962522 8733 leader_election.cc:302] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e13ae747c248482cbcc2359fe3796268; no voters:
I20260811 10:16:10.962738 8733 leader_election.cc:288] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:10.962805 8741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:10.962975 8741 raft_consensus.cc:686] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [term 1 LEADER]: Becoming Leader. State: Replica: e13ae747c248482cbcc2359fe3796268, State: Running, Role: LEADER
I20260811 10:16:10.963047 8733 sys_catalog.cc:564] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:10.963085 8741 consensus_queue.cc:237] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } }
I20260811 10:16:10.963450 8741 sys_catalog.cc:455] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e13ae747c248482cbcc2359fe3796268. Latest consensus state: current_term: 1 leader_uuid: "e13ae747c248482cbcc2359fe3796268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } } }
I20260811 10:16:10.963539 8741 sys_catalog.cc:458] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:10.963469 8742 sys_catalog.cc:455] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e13ae747c248482cbcc2359fe3796268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13ae747c248482cbcc2359fe3796268" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 37903 } } }
I20260811 10:16:10.963696 8742 sys_catalog.cc:458] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268 [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:10.965155 8772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:10.965610 8772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:10.965757 8754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:10.965993 8754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:10.966496 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:10.959517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.8.238.62:37903)
0811 10:16:10.959849 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:10.959858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:10.959894 (+ 36us) server_negotiation.cc:407] Connection header received
0811 10:16:10.960735 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:10.960744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:10.960818 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:10.961084 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:10.961352 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.962853 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.964032 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:10.964617 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:10.964799 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:10.965715 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:10.965719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:10.965719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:10.965944 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:10.965986 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:10.966001 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:10.966068 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:10.966184 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 10:16:10.966270 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":101,"threads_started":1}
I20260811 10:16:10.967638 8754 catalog_manager.cc:1347] Generated new cluster ID: 6ea59e8f79cb4761b7514849d336112d
I20260811 10:16:10.967692 8754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:10.988581 8754 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:10.989282 8754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:10.994480 8754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: Generated new TSK 0
I20260811 10:16:10.994705 8754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:11.003198 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--builtin_ntp_servers=127.8.238.20:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:11.093210 8775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:11.093642 8775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:11.093700 8775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:11.096208 8775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:11.096324 8775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:11.096385 8775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:11.098399 8775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:11.098711 8775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:11.099079 8775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:11.104696 8787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:11.104658 8785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:11.104669 8791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:11.105336 8775 server_base.cc:1034] running on GCE node
I20260811 10:16:11.105593 8775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:11.105898 8775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:11.107122 8775 hybrid_clock.cc:648] HybridClock initialized: now 1786443371107097 us; error 59 us; skew 500 ppm
I20260811 10:16:11.108706 8775 webserver.cc:460] Webserver started at http://127.8.238.1:33267/ using document root <none> and password file <none>
I20260811 10:16:11.108989 8775 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:11.109066 8775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:11.109236 8775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:11.110374 8775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "f9e599ace3834132a1b7fb3b081c2b29"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "08e97a1eb2274c08f92b7d9a892f46ab"
server_key_iv: "f4286601556b75af70882e0b4a3bb8e0"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.110764 8775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e599ace3834132a1b7fb3b081c2b29"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "08e97a1eb2274c08f92b7d9a892f46ab"
server_key_iv: "f4286601556b75af70882e0b4a3bb8e0"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.112438 8775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:16:11.113686 8802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.114070 8775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:11.114236 8775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "f9e599ace3834132a1b7fb3b081c2b29"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "08e97a1eb2274c08f92b7d9a892f46ab"
server_key_iv: "f4286601556b75af70882e0b4a3bb8e0"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.114355 8775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:11.127449 8775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:11.127966 8775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:11.128134 8775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:11.128436 8775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:11.128806 8775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:11.128865 8775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.128902 8775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:11.128945 8775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.135895 8775 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:46583
I20260811 10:16:11.136032 9024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:46583 every 8 connection(s)
I20260811 10:16:11.136327 8775 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:16:11.140084 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 8775
I20260811 10:16:11.140211 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:11.140434 9144 external_mini_cluster.cc:1428] Setting key 22c35034980d6622d30157b0a3056c81
I20260811 10:16:11.141659 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--builtin_ntp_servers=127.8.238.20:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 10:16:11.142696 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.137188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:33261 (local address 127.8.238.62:37903)
0811 10:16:11.137282 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.137285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.137531 (+ 246us) server_negotiation.cc:407] Connection header received
0811 10:16:11.138056 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.138057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.138070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.138125 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:11.138860 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.139463 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.140935 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.141258 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.141355 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.142288 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:11.142289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:11.142290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:11.142296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:11.142314 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:11.142319 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:11.142380 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.142543 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.142581 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 10:16:11.142836 9029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.137165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37903 (local address 127.8.238.1:33261)
0811 10:16:11.137404 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.137440 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.137958 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.138185 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.138188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.138197 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:11.138709 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.138714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.139626 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.139627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.140702 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.140706 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.141341 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.141342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.141476 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.142136 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:11.142137 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:11.142169 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:11.142193 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:11.142445 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:11.142452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:11.142455 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.142526 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.142613 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":125,"threads_started":1}
I20260811 10:16:11.143316 9026 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37903
I20260811 10:16:11.143433 9026 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:11.143667 9026 heartbeater.cc:511] Master 127.8.238.62:37903 requested a full tablet report, sending...
I20260811 10:16:11.144258 8674 ts_manager.cc:194] Registered new tserver with Master: f9e599ace3834132a1b7fb3b081c2b29 (127.8.238.1:46583)
I20260811 10:16:11.144968 8674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.1:33261
W20260811 10:16:11.232383 9033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:11.232601 9033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:11.232623 9033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:11.233682 9033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:11.233717 9033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:11.233740 9033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:11.235038 9033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:11.235246 9033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:11.235512 9033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:11.241750 9041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:11.241844 9033 server_base.cc:1034] running on GCE node
W20260811 10:16:11.241750 9044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:11.242319 9042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:11.242851 9033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:11.243207 9033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:11.244411 9033 hybrid_clock.cc:648] HybridClock initialized: now 1786443371244398 us; error 30 us; skew 500 ppm
I20260811 10:16:11.245967 9033 webserver.cc:460] Webserver started at http://127.8.238.2:34333/ using document root <none> and password file <none>
I20260811 10:16:11.246361 9033 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:11.246459 9033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:11.246634 9033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:11.247843 9033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "2ab2383b8a9e43fd979dd5056c932198"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "47a1cc281eed24b7d3cfc7b6f5d694a7"
server_key_iv: "691a706eee00c47201c691a059d990d5"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.248241 9033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "2ab2383b8a9e43fd979dd5056c932198"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "47a1cc281eed24b7d3cfc7b6f5d694a7"
server_key_iv: "691a706eee00c47201c691a059d990d5"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.249608 9033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:16:11.250495 9053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.250871 9033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:11.251072 9033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "2ab2383b8a9e43fd979dd5056c932198"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "47a1cc281eed24b7d3cfc7b6f5d694a7"
server_key_iv: "691a706eee00c47201c691a059d990d5"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.251219 9033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:11.261250 9033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:11.261632 9033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:11.261801 9033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:11.262071 9033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:11.262482 9033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:11.262552 9033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.262600 9033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:11.262625 9033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.270282 9033 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:37537
I20260811 10:16:11.270380 9274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:37537 every 8 connection(s)
I20260811 10:16:11.270761 9033 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:16:11.277792 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.271984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:43125 (local address 127.8.238.62:37903)
0811 10:16:11.272106 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.272109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.272398 (+ 289us) server_negotiation.cc:407] Connection header received
0811 10:16:11.273055 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.273057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.273070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.273150 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:11.274022 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.274602 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.275877 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.276085 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.276153 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.277378 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:11.277380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:11.277380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:11.277387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:11.277413 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:11.277418 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:11.277490 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.277680 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.277723 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 10:16:11.278034 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.271907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37903 (local address 127.8.238.2:43125)
0811 10:16:11.272168 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.272223 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.272913 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.273247 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.273251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.273262 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:11.273910 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.273915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.274739 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.274741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.275679 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.275683 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.276206 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.276207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.276360 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.277173 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:11.277174 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:11.277211 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:11.277242 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:11.277540 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:11.277549 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:11.277554 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.277645 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.277769 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260811 10:16:11.278978 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9033
I20260811 10:16:11.278987 9275 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37903
I20260811 10:16:11.279109 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:11.279114 9275 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:11.279342 9144 external_mini_cluster.cc:1428] Setting key 6d8be60234c70e9df9e5ed9cdffcbe8d
I20260811 10:16:11.279359 9275 heartbeater.cc:511] Master 127.8.238.62:37903 requested a full tablet report, sending...
I20260811 10:16:11.279872 8673 ts_manager.cc:194] Registered new tserver with Master: 2ab2383b8a9e43fd979dd5056c932198 (127.8.238.2:37537)
I20260811 10:16:11.280404 8673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.2:43125
I20260811 10:16:11.280774 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--builtin_ntp_servers=127.8.238.20:33641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:11.368604 9281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:11.368829 9281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:11.368852 9281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:11.369930 9281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:11.369964 9281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:11.369988 9281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:11.371255 9281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:33641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:37903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:11.371454 9281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:11.371706 9281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:11.377130 9300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:11.377184 9297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:11.377308 9281 server_base.cc:1034] running on GCE node
W20260811 10:16:11.377184 9293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:11.377691 9281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:11.377998 9281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:11.379186 9281 hybrid_clock.cc:648] HybridClock initialized: now 1786443371379176 us; error 32 us; skew 500 ppm
I20260811 10:16:11.380640 9281 webserver.cc:460] Webserver started at http://127.8.238.3:35459/ using document root <none> and password file <none>
I20260811 10:16:11.380990 9281 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:11.381047 9281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:11.381146 9281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:11.382210 9281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "e18def0ecf9b497783fbae2a242dbad5"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "80e79fcae061a6a9c7757878ec2d4a2f"
server_key_iv: "60697cdd6e35778c62a74ada55adc357"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.382529 9281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "e18def0ecf9b497783fbae2a242dbad5"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "80e79fcae061a6a9c7757878ec2d4a2f"
server_key_iv: "60697cdd6e35778c62a74ada55adc357"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.383958 9281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:16:11.384841 9311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.385246 9281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:11.385339 9281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "e18def0ecf9b497783fbae2a242dbad5"
format_stamp: "Formatted at 2026-08-11 10:16:11 on dist-test-slave-7f01"
server_key: "80e79fcae061a6a9c7757878ec2d4a2f"
server_key_iv: "60697cdd6e35778c62a74ada55adc357"
server_key_version: "encryptionkey@0"
I20260811 10:16:11.385403 9281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:11.410444 9281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:11.410777 9281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:11.410895 9281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:11.411139 9281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:11.411576 9281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:11.411643 9281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.411679 9281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:11.411710 9281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:11.418668 9281 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:35663
I20260811 10:16:11.418746 9522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:35663 every 8 connection(s)
I20260811 10:16:11.419093 9281 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:16:11.425355 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.420191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:37553 (local address 127.8.238.62:37903)
0811 10:16:11.420281 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.420284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.420634 (+ 350us) server_negotiation.cc:407] Connection header received
0811 10:16:11.421244 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.421245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.421259 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.421326 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:11.422138 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.422662 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.423705 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.423930 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.424025 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.425002 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:11.425006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:11.425006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:11.425016 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:11.425029 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:11.425033 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:11.425110 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.425240 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.425271 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 10:16:11.425632 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.420145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:37903 (local address 127.8.238.3:37553)
0811 10:16:11.420451 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.420513 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.421126 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.421408 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.421412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.421421 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:11.422033 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.422039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.422766 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.422767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.423591 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.423594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.424041 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.424042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.424183 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.424819 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:11.424819 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:11.424850 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:11.424875 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:11.425115 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:11.425123 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:11.425126 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.425218 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.425355 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":134,"threads_started":1}
I20260811 10:16:11.426113 9523 heartbeater.cc:347] Connected to a master server at 127.8.238.62:37903
I20260811 10:16:11.426210 9523 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:11.426414 9523 heartbeater.cc:511] Master 127.8.238.62:37903 requested a full tablet report, sending...
I20260811 10:16:11.426836 8673 ts_manager.cc:194] Registered new tserver with Master: e18def0ecf9b497783fbae2a242dbad5 (127.8.238.3:35663)
I20260811 10:16:11.427238 8673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.3:37553
I20260811 10:16:11.427978 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9281
I20260811 10:16:11.428087 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:11.428306 9144 external_mini_cluster.cc:1428] Setting key aacdb5e0ca4b8c83ed5f5252c6076005
I20260811 10:16:11.429854 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:11.435159 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.431595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.8.238.62:37903)
0811 10:16:11.431685 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.431688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.431791 (+ 103us) server_negotiation.cc:407] Connection header received
0811 10:16:11.431880 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.431881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.431903 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.431973 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:11.432177 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.432707 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.433573 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.433795 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.433883 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.434843 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:11.434845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:11.434845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:11.434850 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:11.434863 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:11.434868 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:11.434964 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.435058 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.435070 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 10:16:11.437851 8673 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:11.439675 8673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:11.446887 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:37537 (local address 127.0.0.1:43802)
0811 10:16:11.442881 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.442890 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.442932 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.443478 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.443479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.443481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:11.443520 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.443522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444906 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.444907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.445692 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.445694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.446555 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.446556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.446638 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.446652 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.446724 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.446795 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260811 10:16:11.447192 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:35663 (local address 127.0.0.1:41322)
0811 10:16:11.442671 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.442711 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.442779 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.443271 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.443274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.443276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:11.443366 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.443368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444807 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.444807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.446056 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.446059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.447000 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.447001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.447057 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.447062 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.447090 (+ 28us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.447129 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260811 10:16:11.447410 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:46583 (local address 127.0.0.1:42678)
0811 10:16:11.443103 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.443113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.443176 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.443523 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.443525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.443529 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:11.443668 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.443671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444712 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.444713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.446110 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.446113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.447197 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.447198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.447256 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.447262 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.447297 (+ 35us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.447344 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":46,"thread_start_us":105,"threads_started":1}
I20260811 10:16:11.447540 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.8.238.3:35663)
0811 10:16:11.443062 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.443066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.443077 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:11.443125 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.443128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.443144 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.443457 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:11.443516 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444716 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.446152 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.446901 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.447305 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.447395 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.447423 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.447460 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260811 10:16:11.447865 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.8.238.1:46583)
0811 10:16:11.443024 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.443029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.443251 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:16:11.443347 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.443352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.443379 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.443813 (+ 434us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:11.443883 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444607 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.446294 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.447096 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.447539 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.447649 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.447675 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.447719 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260811 10:16:11.448295 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.442638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.8.238.2:37537)
0811 10:16:11.443297 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.443302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.443312 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:11.443353 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.443355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.443371 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.443594 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:11.443648 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.444779 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.445811 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.446441 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.446548 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.446643 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.448161 (+ 1518us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.448210 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":144,"threads_started":1}
I20260811 10:16:11.448637 8902 tablet_service.cc:1467] Processing CreateTablet for tablet d1bd663c060f44a796b69afd4c68e949 (DEFAULT_TABLE table=test-table [id=b4a3f01fc5954f0d94adb6594f3c2688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:11.448614 9399 tablet_service.cc:1467] Processing CreateTablet for tablet d1bd663c060f44a796b69afd4c68e949 (DEFAULT_TABLE table=test-table [id=b4a3f01fc5954f0d94adb6594f3c2688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:11.449049 9399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1bd663c060f44a796b69afd4c68e949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:11.449055 8902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1bd663c060f44a796b69afd4c68e949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:11.451989 9550 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Bootstrap starting.
I20260811 10:16:11.452186 9157 tablet_service.cc:1467] Processing CreateTablet for tablet d1bd663c060f44a796b69afd4c68e949 (DEFAULT_TABLE table=test-table [id=b4a3f01fc5954f0d94adb6594f3c2688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:11.452589 9157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1bd663c060f44a796b69afd4c68e949. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:11.452725 9550 tablet_bootstrap.cc:654] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:11.453083 9550 log.cc:824] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:11.453459 9549 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Bootstrap starting.
I20260811 10:16:11.454174 9549 tablet_bootstrap.cc:654] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:11.454519 9549 log.cc:824] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:11.455116 9550 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: No bootstrap required, opened a new log
I20260811 10:16:11.455207 9550 ts_tablet_manager.cc:1397] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 10:16:11.456143 9549 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: No bootstrap required, opened a new log
I20260811 10:16:11.456235 9549 ts_tablet_manager.cc:1397] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 10:16:11.456615 9555 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Bootstrap starting.
I20260811 10:16:11.457275 9555 tablet_bootstrap.cc:654] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:11.457607 9555 log.cc:824] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:11.458051 9550 raft_consensus.cc:348] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.458252 9550 raft_consensus.cc:374] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:11.458307 9550 raft_consensus.cc:729] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e599ace3834132a1b7fb3b081c2b29, State: Initialized, Role: FOLLOWER
I20260811 10:16:11.458479 9555 tablet_bootstrap.cc:492] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: No bootstrap required, opened a new log
I20260811 10:16:11.458447 9550 consensus_queue.cc:260] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.458549 9555 ts_tablet_manager.cc:1397] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:11.458726 9550 ts_tablet_manager.cc:1428] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:11.458866 9026 heartbeater.cc:503] Master 127.8.238.62:37903 was elected leader, sending a full tablet report...
I20260811 10:16:11.458838 9549 raft_consensus.cc:348] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.459014 9549 raft_consensus.cc:374] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:11.459059 9549 raft_consensus.cc:729] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e18def0ecf9b497783fbae2a242dbad5, State: Initialized, Role: FOLLOWER
I20260811 10:16:11.459175 9549 consensus_queue.cc:260] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.459436 9549 ts_tablet_manager.cc:1428] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:11.459502 9523 heartbeater.cc:503] Master 127.8.238.62:37903 was elected leader, sending a full tablet report...
I20260811 10:16:11.461118 9555 raft_consensus.cc:348] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.461344 9555 raft_consensus.cc:374] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:11.461431 9555 raft_consensus.cc:729] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab2383b8a9e43fd979dd5056c932198, State: Initialized, Role: FOLLOWER
I20260811 10:16:11.461573 9555 consensus_queue.cc:260] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.461863 9555 ts_tablet_manager.cc:1428] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 10:16:11.461896 9275 heartbeater.cc:503] Master 127.8.238.62:37903 was elected leader, sending a full tablet report...
W20260811 10:16:11.521468 9278 tablet.cc:2369] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 10:16:11.637184 9028 tablet.cc:2369] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:11.659396 9560 raft_consensus.cc:482] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:11.659567 9560 raft_consensus.cc:504] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.659965 9560 leader_election.cc:288] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ab2383b8a9e43fd979dd5056c932198 (127.8.238.2:37537), f9e599ace3834132a1b7fb3b081c2b29 (127.8.238.1:46583)
I20260811 10:16:11.664598 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.660325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:35227 (local address 127.8.238.2:37537)
0811 10:16:11.660375 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.660378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.660391 (+ 13us) server_negotiation.cc:407] Connection header received
0811 10:16:11.660462 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.660463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.660474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.660526 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:11.660751 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.661563 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.663415 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.664017 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.664296 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.664327 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.664458 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.664532 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:16:11.664598 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.660196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:37537 (local address 127.8.238.3:35227)
0811 10:16:11.660272 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.660288 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.660374 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.660558 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.660559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.660561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:11.660653 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.660655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.661685 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.661686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.663289 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.663293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.664272 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.664273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.664362 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.664368 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.664448 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.664512 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":113}
I20260811 10:16:11.664945 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.660325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:47695 (local address 127.8.238.1:46583)
0811 10:16:11.660379 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.660382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.660585 (+ 203us) server_negotiation.cc:407] Connection header received
0811 10:16:11.660630 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.660631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.660642 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.660685 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:11.660867 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.661571 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.663415 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.664040 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.664314 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.664336 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.664721 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.664823 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 10:16:11.665199 9187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1bd663c060f44a796b69afd4c68e949" candidate_uuid: "e18def0ecf9b497783fbae2a242dbad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab2383b8a9e43fd979dd5056c932198" is_pre_election: true
I20260811 10:16:11.665287 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.660196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:46583 (local address 127.8.238.3:47695)
0811 10:16:11.660466 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.660477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:11.660566 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:11.660714 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:11.660715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:11.660717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:11.660790 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.660792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.661685 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.661686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.663297 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:11.663298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.664430 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.664432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:11.664561 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.664566 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:11.665121 (+ 555us) client_negotiation.cc:240] Negotiation successful
0811 10:16:11.665188 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":212,"thread_start_us":125,"threads_started":1}
I20260811 10:16:11.665383 9187 raft_consensus.cc:2454] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18def0ecf9b497783fbae2a242dbad5 in term 0.
I20260811 10:16:11.665709 8932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1bd663c060f44a796b69afd4c68e949" candidate_uuid: "e18def0ecf9b497783fbae2a242dbad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e599ace3834132a1b7fb3b081c2b29" is_pre_election: true
I20260811 10:16:11.665745 9313 leader_election.cc:302] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab2383b8a9e43fd979dd5056c932198, e18def0ecf9b497783fbae2a242dbad5; no voters:
I20260811 10:16:11.665844 8932 raft_consensus.cc:2454] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e18def0ecf9b497783fbae2a242dbad5 in term 0.
I20260811 10:16:11.665989 9560 raft_consensus.cc:2781] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:11.666082 9560 raft_consensus.cc:482] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:11.666111 9560 raft_consensus.cc:3037] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:11.667105 9560 raft_consensus.cc:504] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
I20260811 10:16:11.667302 9560 leader_election.cc:288] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [CANDIDATE]: Term 1 election: Requested vote from peers 2ab2383b8a9e43fd979dd5056c932198 (127.8.238.2:37537), f9e599ace3834132a1b7fb3b081c2b29 (127.8.238.1:46583)
I20260811 10:16:11.667483 9187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1bd663c060f44a796b69afd4c68e949" candidate_uuid: "e18def0ecf9b497783fbae2a242dbad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ab2383b8a9e43fd979dd5056c932198"
I20260811 10:16:11.667510 8932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1bd663c060f44a796b69afd4c68e949" candidate_uuid: "e18def0ecf9b497783fbae2a242dbad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e599ace3834132a1b7fb3b081c2b29"
I20260811 10:16:11.667708 8932 raft_consensus.cc:3037] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:11.667821 9187 raft_consensus.cc:3037] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:11.668718 8932 raft_consensus.cc:2454] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18def0ecf9b497783fbae2a242dbad5 in term 1.
I20260811 10:16:11.668854 9187 raft_consensus.cc:2454] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e18def0ecf9b497783fbae2a242dbad5 in term 1.
I20260811 10:16:11.668936 9312 leader_election.cc:302] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e18def0ecf9b497783fbae2a242dbad5, f9e599ace3834132a1b7fb3b081c2b29; no voters:
I20260811 10:16:11.669046 9560 raft_consensus.cc:2781] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:11.669250 9560 raft_consensus.cc:686] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 1 LEADER]: Becoming Leader. State: Replica: e18def0ecf9b497783fbae2a242dbad5, State: Running, Role: LEADER
I20260811 10:16:11.669392 9560 consensus_queue.cc:237] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } }
W20260811 10:16:11.669791 9525 tablet.cc:2369] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:11.670614 8673 catalog_manager.cc:5495] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e18def0ecf9b497783fbae2a242dbad5 (127.8.238.3). New cstate: current_term: 1 leader_uuid: "e18def0ecf9b497783fbae2a242dbad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e18def0ecf9b497783fbae2a242dbad5" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 } health_report { overall_health: UNKNOWN } } }
I20260811 10:16:11.683166 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.680585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.8.238.3:35663)
0811 10:16:11.680633 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:11.680634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:11.680642 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:16:11.680698 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:11.680699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:11.680720 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:11.680783 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:11.680896 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.681374 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.682369 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:11.682559 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:11.682618 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:11.682801 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:11.682892 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:11.682963 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:11.683069 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 10:16:11.683077 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 10:16:11.685431 9187 raft_consensus.cc:1261] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 1 FOLLOWER]: Refusing update from remote peer e18def0ecf9b497783fbae2a242dbad5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:11.685431 8932 raft_consensus.cc:1261] T d1bd663c060f44a796b69afd4c68e949 P f9e599ace3834132a1b7fb3b081c2b29 [term 1 FOLLOWER]: Refusing update from remote peer e18def0ecf9b497783fbae2a242dbad5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:11.685907 9565 consensus_queue.cc:1035] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e599ace3834132a1b7fb3b081c2b29" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:11.686152 9560 consensus_queue.cc:1035] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ab2383b8a9e43fd979dd5056c932198" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 37537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:11.689373 9568 mvcc.cc:205] Tried to move back new op lower bound from 7317272050419855360 to 7317272050359140352. Current Snapshot: MvccSnapshot[applied={T|T < 7317272050419855360}]
I20260811 10:16:11.693181 8674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:11.693318 8674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:11.694489 8674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e13ae747c248482cbcc2359fe3796268: Sending DeleteTablet for 3 replicas of tablet d1bd663c060f44a796b69afd4c68e949
I20260811 10:16:11.695191 9399 tablet_service.cc:1514] Processing DeleteTablet for tablet d1bd663c060f44a796b69afd4c68e949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:11 UTC) from {username='slave'} at 127.0.0.1:41322
I20260811 10:16:11.695199 9157 tablet_service.cc:1514] Processing DeleteTablet for tablet d1bd663c060f44a796b69afd4c68e949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:11 UTC) from {username='slave'} at 127.0.0.1:43802
I20260811 10:16:11.695665 9582 tablet_replica.cc:330] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: stopping tablet replica
I20260811 10:16:11.695729 8902 tablet_service.cc:1514] Processing DeleteTablet for tablet d1bd663c060f44a796b69afd4c68e949 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:11 UTC) from {username='slave'} at 127.0.0.1:42678
I20260811 10:16:11.695820 9582 raft_consensus.cc:2229] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:11.695984 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 8775
I20260811 10:16:11.696069 9583 tablet_replica.cc:330] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: stopping tablet replica
I20260811 10:16:11.712445 9583 raft_consensus.cc:2229] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:16:11.712706 9583 raft_consensus.cc:2258] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:11.713949 9583 ts_tablet_manager.cc:1905] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:11.716048 9583 ts_tablet_manager.cc:1918] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:11.716120 9583 log.cc:1196] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/d1bd663c060f44a796b69afd4c68e949
I20260811 10:16:11.716392 9583 ts_tablet_manager.cc:1939] T d1bd663c060f44a796b69afd4c68e949 P e18def0ecf9b497783fbae2a242dbad5: Deleting consensus metadata
I20260811 10:16:11.717710 8640 catalog_manager.cc:4841] TS e18def0ecf9b497783fbae2a242dbad5 (127.8.238.3:35663): tablet d1bd663c060f44a796b69afd4c68e949 (table test-table [id=b4a3f01fc5954f0d94adb6594f3c2688]) successfully deleted
I20260811 10:16:11.718261 9582 raft_consensus.cc:2258] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:11.718822 9582 ts_tablet_manager.cc:1905] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:11.720911 9582 ts_tablet_manager.cc:1918] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:11.721060 9582 log.cc:1196] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/d1bd663c060f44a796b69afd4c68e949
I20260811 10:16:11.721361 9582 ts_tablet_manager.cc:1939] T d1bd663c060f44a796b69afd4c68e949 P 2ab2383b8a9e43fd979dd5056c932198: Deleting consensus metadata
I20260811 10:16:11.721979 8641 catalog_manager.cc:4841] TS 2ab2383b8a9e43fd979dd5056c932198 (127.8.238.2:37537): tablet d1bd663c060f44a796b69afd4c68e949 (table test-table [id=b4a3f01fc5954f0d94adb6594f3c2688]) successfully deleted
W20260811 10:16:11.724318 8640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:46583 (error 108)
I20260811 10:16:11.724537 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9033
I20260811 10:16:11.724936 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:11.724799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:46583 (local address 127.0.0.1:42686)
0811 10:16:11.724855 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:11.724891 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:16:11.725049 8640 catalog_manager.cc:4584] TS f9e599ace3834132a1b7fb3b081c2b29 (127.8.238.1:46583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1bd663c060f44a796b69afd4c68e949: Network error: Client connection negotiation failed: client connection to 127.8.238.1:46583: connect: Connection refused (error 111)
I20260811 10:16:11.734942 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9281
I20260811 10:16:11.746924 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 8617
2026-08-11T10:16:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (970 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:11 dist-test-slave-7f01 krb5kdc[9599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:11 dist-test-slave-7f01 krb5kdc[9599](info): set up 2 sockets
Aug 11 10:16:11 dist-test-slave-7f01 krb5kdc[9599](info): commencing operation
krb5kdc: starting...
W20260811 10:16:13.097779 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:13.256672 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43735
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:43735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:13.358218 9628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:13.358505 9628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:13.358544 9628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:13.359730 9628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:13.359839 9628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:13.359865 9628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:13.359894 9628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:13.359923 9628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:13.361472 9628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:43735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:13.361781 9628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:13.362200 9628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:13.367439 9644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.367439 9641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.367595 9628 server_base.cc:1034] running on GCE node
W20260811 10:16:13.367434 9640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.368063 9628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:13.368429 9628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:13.369618 9628 hybrid_clock.cc:648] HybridClock initialized: now 1786443373369611 us; error 39 us; skew 500 ppm
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:13.373420 9628 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:13.373992 9628 webserver.cc:460] Webserver started at http://127.8.238.62:43031/ using document root <none> and password file <none>
I20260811 10:16:13.374310 9628 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:13.374366 9628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:13.374524 9628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:13.375926 9628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "552a562e24a44e4cb72f193639def70c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "901e9985d311b01e5d08207c6fe207fd"
server_key_iv: "a0bb114353959da6ddd965760aabfb3a"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.376374 9628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "552a562e24a44e4cb72f193639def70c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "901e9985d311b01e5d08207c6fe207fd"
server_key_iv: "a0bb114353959da6ddd965760aabfb3a"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.378150 9628 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:13.379366 9653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.379699 9628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:13.379922 9628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "552a562e24a44e4cb72f193639def70c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "901e9985d311b01e5d08207c6fe207fd"
server_key_iv: "a0bb114353959da6ddd965760aabfb3a"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.380183 9628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:13.388882 9628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:13.391424 9628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:13.391603 9628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:13.396075 9628 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:43735
I20260811 10:16:13.396137 9748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:43735 every 8 connection(s)
I20260811 10:16:13.396548 9628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:13.397930 9749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:13.400943 9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: Bootstrap starting.
I20260811 10:16:13.401897 9749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:13.402277 9749 log.cc:824] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:13.403435 9749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: No bootstrap required, opened a new log
I20260811 10:16:13.405469 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9628
I20260811 10:16:13.405624 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:13.405834 9144 external_mini_cluster.cc:1428] Setting key ba34b3aff93b9a3477220a5645c82dd7
I20260811 10:16:13.405827 9749 raft_consensus.cc:348] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } }
I20260811 10:16:13.406064 9749 raft_consensus.cc:374] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:13.406131 9749 raft_consensus.cc:729] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552a562e24a44e4cb72f193639def70c, State: Initialized, Role: FOLLOWER
I20260811 10:16:13.406325 9749 consensus_queue.cc:260] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } }
I20260811 10:16:13.406428 9749 raft_consensus.cc:388] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:13.406469 9749 raft_consensus.cc:482] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:13.406535 9749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:13.407259 9749 raft_consensus.cc:504] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } }
I20260811 10:16:13.407357 9749 leader_election.cc:302] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 552a562e24a44e4cb72f193639def70c; no voters:
I20260811 10:16:13.407580 9749 leader_election.cc:288] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:13.407603 9755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:13.407752 9755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [term 1 LEADER]: Becoming Leader. State: Replica: 552a562e24a44e4cb72f193639def70c, State: Running, Role: LEADER
I20260811 10:16:13.407836 9755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } }
I20260811 10:16:13.407907 9749 sys_catalog.cc:564] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:13.408193 9755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552a562e24a44e4cb72f193639def70c. Latest consensus state: current_term: 1 leader_uuid: "552a562e24a44e4cb72f193639def70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } } }
I20260811 10:16:13.408268 9755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:13.408438 9756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "552a562e24a44e4cb72f193639def70c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a562e24a44e4cb72f193639def70c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43735 } } }
I20260811 10:16:13.408524 9756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:13.408669 9759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:13.409475 9759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:13.411177 9759 catalog_manager.cc:1347] Generated new cluster ID: 9b47e6508569456caa37f0b5a148d594
I20260811 10:16:13.411234 9759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:13.419451 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:13.406412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.8.238.62:43735)
0811 10:16:13.406783 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:13.406793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:13.406826 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:16:13.407221 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:13.407229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:13.407277 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:13.407475 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:13.408364 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.409874 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.410959 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.411564 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.411676 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:13.414831 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:13.414836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:13.414836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:13.414849 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:13.417191 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.417903 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.417907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.417907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.418060 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.418341 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.418342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.418342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.418678 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:13.418855 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:13.419046 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 10:16:13.419209 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":112,"threads_started":1}
I20260811 10:16:13.421954 9759 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:13.422499 9759 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:13.428922 9759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: Generated new TSK 0
I20260811 10:16:13.429138 9759 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.238.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
Entry for principal kudu/127.8.238.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab.
I20260811 10:16:13.472002 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:13.567230 9794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:13.567462 9794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:13.567494 9794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:13.568883 9794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:13.568933 9794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:13.568964 9794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:13.570480 9794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.1.keytab
--principal=kudu/127.8.238.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:13.570699 9794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:13.570909 9794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:13.576644 9805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.576651 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.576651 9806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.577155 9794 server_base.cc:1034] running on GCE node
I20260811 10:16:13.577414 9794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:13.577692 9794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:13.578890 9794 hybrid_clock.cc:648] HybridClock initialized: now 1786443373578856 us; error 59 us; skew 500 ppm
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:13.581933 9794 init.cc:377] Logged in from keytab as kudu/127.8.238.1@KRBTEST.COM (short username kudu)
I20260811 10:16:13.582458 9794 webserver.cc:460] Webserver started at http://127.8.238.1:34221/ using document root <none> and password file <none>
I20260811 10:16:13.582674 9794 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:13.582718 9794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:13.582866 9794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:13.584151 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "becd03cbb32341a58b944f551a5a76b0"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "ba0549ab7539be1f3bd2257e6864d4c6"
server_key_iv: "6065d7ae79716a283235c04ba2d277ac"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.584542 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "becd03cbb32341a58b944f551a5a76b0"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "ba0549ab7539be1f3bd2257e6864d4c6"
server_key_iv: "6065d7ae79716a283235c04ba2d277ac"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.585965 9794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:13.586827 9825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.587145 9794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:13.587272 9794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "becd03cbb32341a58b944f551a5a76b0"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "ba0549ab7539be1f3bd2257e6864d4c6"
server_key_iv: "6065d7ae79716a283235c04ba2d277ac"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.587366 9794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:13.599504 9794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:13.602639 9794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:13.602864 9794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:13.603152 9794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:13.603534 9794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:13.603605 9794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.603652 9794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:13.603682 9794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.610975 9794 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:39741
I20260811 10:16:13.611063 10035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:39741 every 8 connection(s)
I20260811 10:16:13.611445 9794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.1@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:13.619194 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 9794
I20260811 10:16:13.619351 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:13.619649 9144 external_mini_cluster.cc:1428] Setting key 902f63815f13943511f80f54424efeec
I20260811 10:16:13.624468 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:13.612523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:55489 (local address 127.8.238.62:43735)
0811 10:16:13.612622 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:13.612625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:13.612644 (+ 19us) server_negotiation.cc:407] Connection header received
0811 10:16:13.613250 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:13.613252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:13.613271 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:13.613368 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:13.614459 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.615020 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.616613 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.616850 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.616939 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:13.620143 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:13.620145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:13.620145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:13.620153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:13.622256 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.622949 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.622951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.622952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.623039 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.623420 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.623421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.623421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.623553 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:13.623626 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:13.624274 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 10:16:13.624389 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 10:16:13.624948 10039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:13.612313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43735 (local address 127.8.238.1:55489)
0811 10:16:13.612514 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:13.612559 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:13.613098 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:13.613463 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:13.613467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:13.614078 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:13.614298 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:13.614303 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.615267 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.615269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:13.616464 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:13.616469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.617001 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.617003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:13.617174 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:13.617851 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:13.617866 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:13.619957 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:13.622391 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:13.622401 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:13.622404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:13.622820 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:13.623125 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:13.623126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:13.623126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:13.623338 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:13.623644 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:13.623651 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:13.624030 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 10:16:13.624202 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 10:16:13.624444 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260811 10:16:13.625705 10036 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43735
I20260811 10:16:13.625826 10036 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:13.626073 10036 heartbeater.cc:511] Master 127.8.238.62:43735 requested a full tablet report, sending...
I20260811 10:16:13.626694 9683 ts_manager.cc:194] Registered new tserver with Master: becd03cbb32341a58b944f551a5a76b0 (127.8.238.1:39741)
I20260811 10:16:13.627534 9683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.1@KRBTEST.COM'} at 127.8.238.1:55489
WARNING: no policy specified for kudu/127.8.238.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
Entry for principal kudu/127.8.238.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab.
I20260811 10:16:13.656773 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:13.750319 10043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:13.750697 10043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:13.750746 10043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:13.751994 10043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:13.752066 10043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:13.752115 10043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:13.753679 10043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.2.keytab
--principal=kudu/127.8.238.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:13.753983 10043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:13.754242 10043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:13.760068 10050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.760093 10051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.760205 10043 server_base.cc:1034] running on GCE node
W20260811 10:16:13.760093 10053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.760632 10043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:13.760951 10043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:13.762141 10043 hybrid_clock.cc:648] HybridClock initialized: now 1786443373762112 us; error 39 us; skew 500 ppm
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:13.765444 10043 init.cc:377] Logged in from keytab as kudu/127.8.238.2@KRBTEST.COM (short username kudu)
I20260811 10:16:13.766000 10043 webserver.cc:460] Webserver started at http://127.8.238.2:45023/ using document root <none> and password file <none>
I20260811 10:16:13.766245 10043 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:13.766314 10043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:13.766479 10043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:13.767617 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "8a44d462f2234fe99b9dc15471d8664c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "3905ccc3d72bc17890f9ffbb5e08c5e4"
server_key_iv: "084173a76f4654b0c0839c64966b319f"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.768081 10043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "8a44d462f2234fe99b9dc15471d8664c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "3905ccc3d72bc17890f9ffbb5e08c5e4"
server_key_iv: "084173a76f4654b0c0839c64966b319f"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.769820 10043 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:13.770949 10067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.771337 10043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:13.771518 10043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "8a44d462f2234fe99b9dc15471d8664c"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "3905ccc3d72bc17890f9ffbb5e08c5e4"
server_key_iv: "084173a76f4654b0c0839c64966b319f"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.771642 10043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:13.802294 10043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:13.805356 10043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:13.805610 10043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:13.805941 10043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:13.806494 10043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:13.806581 10043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.806633 10043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:13.806691 10043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.815168 10043 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:43401
I20260811 10:16:13.815263 10296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:43401 every 8 connection(s)
I20260811 10:16:13.815853 10043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.2@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:13.825742 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10043
I20260811 10:16:13.825922 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:13.826264 9144 external_mini_cluster.cc:1428] Setting key 132fe6e9fd01eb52bad3d5917422efce
I20260811 10:16:13.827248 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:13.817017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:42281 (local address 127.8.238.62:43735)
0811 10:16:13.817092 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:13.817095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:13.817289 (+ 194us) server_negotiation.cc:407] Connection header received
0811 10:16:13.817920 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:13.817922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:13.817936 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:13.817984 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:13.819109 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.819641 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.820742 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.820955 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.821036 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:13.823813 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:13.823820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:13.823820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:13.823828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:13.825473 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.826096 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.826097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.826098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.826190 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:13.826519 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:13.826520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:13.826520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:13.826652 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:13.826734 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:13.827053 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 10:16:13.827146 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:16:13.827495 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:13.816896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43735 (local address 127.8.238.2:42281)
0811 10:16:13.817112 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:13.817175 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:13.817777 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:13.818077 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:13.818081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:13.818764 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:13.818998 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:13.819003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.819789 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.819790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:13.820639 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:13.820642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:13.821059 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:13.821061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:13.821207 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:13.821873 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:13.821886 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:13.823661 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:13.825682 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:13.825692 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:13.825697 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:13.826020 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:13.826272 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:13.826273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:13.826273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:13.826448 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:13.826724 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:13.826733 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:13.826910 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 10:16:13.827032 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 10:16:13.827221 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260811 10:16:13.828017 10298 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43735
I20260811 10:16:13.828140 10298 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:13.828363 10298 heartbeater.cc:511] Master 127.8.238.62:43735 requested a full tablet report, sending...
I20260811 10:16:13.828792 9683 ts_manager.cc:194] Registered new tserver with Master: 8a44d462f2234fe99b9dc15471d8664c (127.8.238.2:43401)
I20260811 10:16:13.829327 9683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.2@KRBTEST.COM'} at 127.8.238.2:42281
WARNING: no policy specified for kudu/127.8.238.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
Entry for principal kudu/127.8.238.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab.
I20260811 10:16:13.861104 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:13.958421 10308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:13.958709 10308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:13.958755 10308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:13.960188 10308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:13.960266 10308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:13.960320 10308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:13.962116 10308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.3.keytab
--principal=kudu/127.8.238.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:13.962455 10308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:13.962746 10308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:13.968693 10318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.968685 10323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:13.968755 10320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:13.968953 10308 server_base.cc:1034] running on GCE node
I20260811 10:16:13.969185 10308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:13.969511 10308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:13.970724 10308 hybrid_clock.cc:648] HybridClock initialized: now 1786443373970719 us; error 44 us; skew 500 ppm
Aug 11 10:16:13 dist-test-slave-7f01 krb5kdc[9599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:13.973958 10308 init.cc:377] Logged in from keytab as kudu/127.8.238.3@KRBTEST.COM (short username kudu)
I20260811 10:16:13.974589 10308 webserver.cc:460] Webserver started at http://127.8.238.3:33623/ using document root <none> and password file <none>
I20260811 10:16:13.974896 10308 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:13.974978 10308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:13.975129 10308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:13.976574 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "7b949e362bb842caa0bbc949d7cc981b"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "f0d1ac1353037a10b95aba77bdebb19e"
server_key_iv: "401de66eb8aa4efe8f4dbe621d37f2d8"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.976915 10308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "7b949e362bb842caa0bbc949d7cc981b"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "f0d1ac1353037a10b95aba77bdebb19e"
server_key_iv: "401de66eb8aa4efe8f4dbe621d37f2d8"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.978365 10308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:13.979200 10338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:13.979385 10308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:13.979494 10308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "7b949e362bb842caa0bbc949d7cc981b"
format_stamp: "Formatted at 2026-08-11 10:16:13 on dist-test-slave-7f01"
server_key: "f0d1ac1353037a10b95aba77bdebb19e"
server_key_iv: "401de66eb8aa4efe8f4dbe621d37f2d8"
server_key_version: "encryptionkey@0"
I20260811 10:16:13.979568 10308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:13.996776 10308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:13.999399 10308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:13.999538 10308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:13.999787 10308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:14.000098 10308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:14.000134 10308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.000159 10308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:14.000172 10308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.006989 10308 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:45799
I20260811 10:16:14.007151 10521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:45799 every 8 connection(s)
I20260811 10:16:14.007467 10308 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:16:14.008759 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10308
I20260811 10:16:14.008870 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:14.009078 9144 external_mini_cluster.cc:1428] Setting key dafb86397929503a9370905d97c19bb4
Aug 11 10:16:14 dist-test-slave-7f01 krb5kdc[9599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443373, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.3@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:14.020210 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.008703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:34273 (local address 127.8.238.62:43735)
0811 10:16:14.008769 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.008771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.008991 (+ 220us) server_negotiation.cc:407] Connection header received
0811 10:16:14.009902 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.009905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.009928 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.010011 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.011238 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.011853 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.013003 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.013345 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.013446 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.016838 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.016840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.016840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:14.016852 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.018519 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:14.019103 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:14.019104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:14.019104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:14.019176 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:14.019451 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:14.019452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:14.019452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:14.019593 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.019650 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.020038 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.020128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:16:14.020437 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.008583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:43735 (local address 127.8.238.3:34273)
0811 10:16:14.008808 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.008881 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.009682 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.010119 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.010124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.010865 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:14.011086 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.011091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.011952 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.011953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.012800 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.012802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.013543 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.013545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.013742 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.014597 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:14.014610 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:14.016667 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:14.018642 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:14.018657 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:14.018660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:14.019000 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:14.019258 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:14.019259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:14.019259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:14.019387 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:14.019654 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:14.019659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:14.019893 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.019992 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.020157 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":114,"threads_started":1}
I20260811 10:16:14.021394 10522 heartbeater.cc:347] Connected to a master server at 127.8.238.62:43735
I20260811 10:16:14.021543 10522 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:14.021807 10522 heartbeater.cc:511] Master 127.8.238.62:43735 requested a full tablet report, sending...
I20260811 10:16:14.022387 9682 ts_manager.cc:194] Registered new tserver with Master: 7b949e362bb842caa0bbc949d7cc981b (127.8.238.3:45799)
I20260811 10:16:14.022818 9682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.238.3@KRBTEST.COM'} at 127.8.238.3:34273
I20260811 10:16:14.033771 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:14.043347 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.035574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.8.238.62:43735)
0811 10:16:14.035693 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.035696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.035871 (+ 175us) server_negotiation.cc:407] Connection header received
0811 10:16:14.035972 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.035974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.035988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.036101 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.036827 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.037376 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.038145 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.038353 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.038443 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.040170 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.040175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.040175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:14.040190 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.041859 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:14.042378 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:14.042379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:14.042379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:14.042444 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:14.042660 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:14.042661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:14.042661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:14.042853 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.042976 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.043131 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.043231 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 10:16:14.045745 9682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:14.047590 9682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:14.055667 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:39741 (local address 127.0.0.1:48966)
0811 10:16:14.051470 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.051482 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.051576 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.051845 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.051846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.051848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:14.051922 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.051924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.052925 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.052926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.054340 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.054343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055264 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.055361 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055379 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.055482 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.055559 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":31,"spinlock_wait_cycles":896,"thread_start_us":161,"threads_started":1}
I20260811 10:16:14.055670 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.8.238.1:39741)
0811 10:16:14.051224 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.051230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.051565 (+ 335us) server_negotiation.cc:407] Connection header received
0811 10:16:14.051660 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.051664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.051685 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.051812 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:14.051993 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.052794 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.054462 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055161 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055231 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055283 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.055501 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.055554 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
I20260811 10:16:14.055812 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.8.238.2:43401)
0811 10:16:14.051321 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.051325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.051791 (+ 466us) server_negotiation.cc:407] Connection header received
0811 10:16:14.051856 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.051860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.051885 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.051984 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:14.052129 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.053338 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.054662 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055190 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055290 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.055616 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.055669 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260811 10:16:14.056103 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:45799 (local address 127.0.0.1:55396)
0811 10:16:14.050698 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.050739 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.050815 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.051207 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.051210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.051213 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:14.051297 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.051299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.052875 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.052877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.054118 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.054121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055255 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.055361 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055379 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.055475 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.055552 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":97,"thread_start_us":70,"threads_started":1}
I20260811 10:16:14.056106 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:43401 (local address 127.0.0.1:43306)
0811 10:16:14.051671 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.051681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.051770 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.052015 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.052015 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.052017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:14.052060 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.052061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.053475 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.053476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.054564 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.054567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055393 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.055451 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055454 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.055502 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.055553 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1090,"mutex_wait_us":4,"thread_start_us":156,"threads_started":1}
I20260811 10:16:14.056324 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.050614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.8.238.3:45799)
0811 10:16:14.050868 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.050873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.050884 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:14.050929 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.050931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.050946 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.051111 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:14.051429 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.052721 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.054220 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.055047 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.055129 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.055212 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.056039 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.056105 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":137,"threads_started":1}
I20260811 10:16:14.057201 10170 tablet_service.cc:1467] Processing CreateTablet for tablet a635ab5e12da4eacada29f98a92270b5 (DEFAULT_TABLE table=test-table [id=6c32317af2b24382aed5c8efce9fa07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:14.057202 10416 tablet_service.cc:1467] Processing CreateTablet for tablet a635ab5e12da4eacada29f98a92270b5 (DEFAULT_TABLE table=test-table [id=6c32317af2b24382aed5c8efce9fa07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:14.057253 9902 tablet_service.cc:1467] Processing CreateTablet for tablet a635ab5e12da4eacada29f98a92270b5 (DEFAULT_TABLE table=test-table [id=6c32317af2b24382aed5c8efce9fa07c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:14.057677 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a635ab5e12da4eacada29f98a92270b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:14.057677 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a635ab5e12da4eacada29f98a92270b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:14.057677 10416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a635ab5e12da4eacada29f98a92270b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:14.061120 10555 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Bootstrap starting.
I20260811 10:16:14.061599 10559 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Bootstrap starting.
I20260811 10:16:14.061905 10556 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Bootstrap starting.
I20260811 10:16:14.062476 10559 tablet_bootstrap.cc:654] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:14.062651 10556 tablet_bootstrap.cc:654] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:14.062918 10555 tablet_bootstrap.cc:654] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:14.063267 10559 log.cc:824] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:14.063338 10556 log.cc:824] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:14.063691 10555 log.cc:824] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:14.064669 10559 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: No bootstrap required, opened a new log
I20260811 10:16:14.064777 10555 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: No bootstrap required, opened a new log
I20260811 10:16:14.064766 10556 tablet_bootstrap.cc:492] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: No bootstrap required, opened a new log
I20260811 10:16:14.064790 10559 ts_tablet_manager.cc:1397] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:16:14.064862 10555 ts_tablet_manager.cc:1397] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 10:16:14.064862 10556 ts_tablet_manager.cc:1397] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 10:16:14.066423 10559 raft_consensus.cc:348] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.066715 10559 raft_consensus.cc:374] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:14.066788 10559 raft_consensus.cc:729] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a44d462f2234fe99b9dc15471d8664c, State: Initialized, Role: FOLLOWER
I20260811 10:16:14.066965 10559 consensus_queue.cc:260] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.067075 10556 raft_consensus.cc:348] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.067266 10556 raft_consensus.cc:374] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:14.067327 10556 raft_consensus.cc:729] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b949e362bb842caa0bbc949d7cc981b, State: Initialized, Role: FOLLOWER
I20260811 10:16:14.067334 10559 ts_tablet_manager.cc:1428] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:14.067448 10556 consensus_queue.cc:260] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.067502 10298 heartbeater.cc:503] Master 127.8.238.62:43735 was elected leader, sending a full tablet report...
I20260811 10:16:14.067610 10555 raft_consensus.cc:348] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.067800 10556 ts_tablet_manager.cc:1428] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:14.067816 10555 raft_consensus.cc:374] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:14.067827 10522 heartbeater.cc:503] Master 127.8.238.62:43735 was elected leader, sending a full tablet report...
I20260811 10:16:14.067866 10555 raft_consensus.cc:729] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: becd03cbb32341a58b944f551a5a76b0, State: Initialized, Role: FOLLOWER
I20260811 10:16:14.068013 10555 consensus_queue.cc:260] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.068288 10555 ts_tablet_manager.cc:1428] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 10:16:14.068523 10036 heartbeater.cc:503] Master 127.8.238.62:43735 was elected leader, sending a full tablet report...
W20260811 10:16:14.112380 10038 tablet.cc:2369] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:14.211527 10571 raft_consensus.cc:482] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:14.211866 10571 raft_consensus.cc:504] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.212538 10571 leader_election.cc:288] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers becd03cbb32341a58b944f551a5a76b0 (127.8.238.1:39741), 8a44d462f2234fe99b9dc15471d8664c (127.8.238.2:43401)
I20260811 10:16:14.216410 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.212866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:39741 (local address 127.8.238.3:57817)
0811 10:16:14.212954 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.212971 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.213035 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.213384 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.213387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.213399 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:14.213529 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.213532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.214352 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.214354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.215200 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.215203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.216003 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.216005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.216126 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.216133 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.216243 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.216323 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 10:16:14.216437 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.212949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:57817 (local address 127.8.238.1:39741)
0811 10:16:14.213012 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.213016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.213032 (+ 16us) server_negotiation.cc:407] Connection header received
0811 10:16:14.213185 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.213187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.213207 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.213272 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:14.213654 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.214236 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.215351 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.215895 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.215964 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.215978 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.216278 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.216328 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 10:16:14.216630 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.212886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:43401 (local address 127.8.238.3:36115)
0811 10:16:14.213155 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.213168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.213244 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.213524 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.213526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.213528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:14.213662 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.213664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.214463 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.214465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.215322 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.215326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.216307 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.216308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.216387 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.216393 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.216468 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.216523 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":4,"thread_start_us":189,"threads_started":1}
I20260811 10:16:14.216660 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.213046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:36115 (local address 127.8.238.2:43401)
0811 10:16:14.213121 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.213123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.213330 (+ 207us) server_negotiation.cc:407] Connection header received
0811 10:16:14.213381 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.213383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.213406 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.213485 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:14.213762 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.214316 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.215492 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.216183 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.216274 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.216289 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.216505 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.216568 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:16:14.216974 9942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a635ab5e12da4eacada29f98a92270b5" candidate_uuid: "7b949e362bb842caa0bbc949d7cc981b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becd03cbb32341a58b944f551a5a76b0" is_pre_election: true
I20260811 10:16:14.217206 9942 raft_consensus.cc:2454] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b949e362bb842caa0bbc949d7cc981b in term 0.
I20260811 10:16:14.217187 10212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a635ab5e12da4eacada29f98a92270b5" candidate_uuid: "7b949e362bb842caa0bbc949d7cc981b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a44d462f2234fe99b9dc15471d8664c" is_pre_election: true
I20260811 10:16:14.217336 10212 raft_consensus.cc:2454] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b949e362bb842caa0bbc949d7cc981b in term 0.
I20260811 10:16:14.217557 10340 leader_election.cc:302] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b949e362bb842caa0bbc949d7cc981b, becd03cbb32341a58b944f551a5a76b0; no voters:
I20260811 10:16:14.217746 10571 raft_consensus.cc:2781] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:14.217826 10571 raft_consensus.cc:482] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:14.217857 10571 raft_consensus.cc:3037] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:14.218888 10571 raft_consensus.cc:504] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.219098 10571 leader_election.cc:288] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [CANDIDATE]: Term 1 election: Requested vote from peers becd03cbb32341a58b944f551a5a76b0 (127.8.238.1:39741), 8a44d462f2234fe99b9dc15471d8664c (127.8.238.2:43401)
I20260811 10:16:14.219403 10212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a635ab5e12da4eacada29f98a92270b5" candidate_uuid: "7b949e362bb842caa0bbc949d7cc981b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a44d462f2234fe99b9dc15471d8664c"
I20260811 10:16:14.219419 9942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a635ab5e12da4eacada29f98a92270b5" candidate_uuid: "7b949e362bb842caa0bbc949d7cc981b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "becd03cbb32341a58b944f551a5a76b0"
I20260811 10:16:14.219525 9942 raft_consensus.cc:3037] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:14.219525 10212 raft_consensus.cc:3037] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:14.220686 10212 raft_consensus.cc:2454] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b949e362bb842caa0bbc949d7cc981b in term 1.
I20260811 10:16:14.220686 9942 raft_consensus.cc:2454] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b949e362bb842caa0bbc949d7cc981b in term 1.
I20260811 10:16:14.220957 10340 leader_election.cc:302] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b949e362bb842caa0bbc949d7cc981b, becd03cbb32341a58b944f551a5a76b0; no voters:
I20260811 10:16:14.221175 10571 raft_consensus.cc:2781] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:14.221418 10571 raft_consensus.cc:686] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 1 LEADER]: Becoming Leader. State: Replica: 7b949e362bb842caa0bbc949d7cc981b, State: Running, Role: LEADER
I20260811 10:16:14.221553 10571 consensus_queue.cc:237] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } }
I20260811 10:16:14.222710 9683 catalog_manager.cc:5495] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b949e362bb842caa0bbc949d7cc981b (127.8.238.3). New cstate: current_term: 1 leader_uuid: "7b949e362bb842caa0bbc949d7cc981b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b949e362bb842caa0bbc949d7cc981b" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 45799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 } health_report { overall_health: UNKNOWN } } }
I20260811 10:16:14.241443 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.238135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55398 (local address 127.8.238.3:45799)
0811 10:16:14.238235 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.238238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.238249 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:14.238296 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.238297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.238325 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.238404 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:14.238671 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.239244 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.240323 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.240572 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.240655 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.240968 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:14.241105 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:14.241189 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.241278 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.241305 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 10:16:14.243889 10212 raft_consensus.cc:1261] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 1 FOLLOWER]: Refusing update from remote peer 7b949e362bb842caa0bbc949d7cc981b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:14.243965 9942 raft_consensus.cc:1261] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 1 FOLLOWER]: Refusing update from remote peer 7b949e362bb842caa0bbc949d7cc981b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:14.244403 10575 consensus_queue.cc:1035] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [LEADER]: Connected to new peer: Peer: permanent_uuid: "becd03cbb32341a58b944f551a5a76b0" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 39741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:14.244525 10571 consensus_queue.cc:1035] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a44d462f2234fe99b9dc15471d8664c" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 43401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:14.247191 10579 mvcc.cc:205] Tried to move back new op lower bound from 7317272060899065856 to 7317272060812505088. Current Snapshot: MvccSnapshot[applied={T|T < 7317272060899065856}]
I20260811 10:16:14.247458 10581 mvcc.cc:205] Tried to move back new op lower bound from 7317272060899065856 to 7317272060812505088. Current Snapshot: MvccSnapshot[applied={T|T < 7317272060899065856}]
I20260811 10:16:14.250773 9682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:14.250918 9682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:14.251837 9682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 552a562e24a44e4cb72f193639def70c: Sending DeleteTablet for 3 replicas of tablet a635ab5e12da4eacada29f98a92270b5
I20260811 10:16:14.252271 9902 tablet_service.cc:1514] Processing DeleteTablet for tablet a635ab5e12da4eacada29f98a92270b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:14 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:48966
I20260811 10:16:14.252310 10170 tablet_service.cc:1514] Processing DeleteTablet for tablet a635ab5e12da4eacada29f98a92270b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:14 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:43306
I20260811 10:16:14.252359 10416 tablet_service.cc:1514] Processing DeleteTablet for tablet a635ab5e12da4eacada29f98a92270b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:14 UTC) from {username='kudu', principal='kudu/127.8.238.62@KRBTEST.COM'} at 127.0.0.1:55396
I20260811 10:16:14.252619 10598 tablet_replica.cc:330] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: stopping tablet replica
I20260811 10:16:14.252696 10597 tablet_replica.cc:330] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0: stopping tablet replica
I20260811 10:16:14.252821 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9794
I20260811 10:16:14.252821 10598 raft_consensus.cc:2229] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:14.252830 10597 raft_consensus.cc:2229] T a635ab5e12da4eacada29f98a92270b5 P becd03cbb32341a58b944f551a5a76b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:14.258396 10598 raft_consensus.cc:2258] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:14.262406 10598 ts_tablet_manager.cc:1905] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:14.264672 10598 ts_tablet_manager.cc:1918] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:14.264868 10598 log.cc:1196] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/a635ab5e12da4eacada29f98a92270b5
I20260811 10:16:14.265245 10598 ts_tablet_manager.cc:1939] T a635ab5e12da4eacada29f98a92270b5 P 8a44d462f2234fe99b9dc15471d8664c: Deleting consensus metadata
I20260811 10:16:14.266916 9655 catalog_manager.cc:4841] TS 8a44d462f2234fe99b9dc15471d8664c (127.8.238.2:43401): tablet a635ab5e12da4eacada29f98a92270b5 (table test-table [id=6c32317af2b24382aed5c8efce9fa07c]) successfully deleted
I20260811 10:16:14.267033 10599 tablet_replica.cc:330] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: stopping tablet replica
I20260811 10:16:14.267155 10599 raft_consensus.cc:2229] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 1 LEADER]: Raft consensus shutting down.
I20260811 10:16:14.267324 10599 raft_consensus.cc:2258] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:14.269142 10599 ts_tablet_manager.cc:1905] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:14.271031 10599 ts_tablet_manager.cc:1918] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:14.271126 10599 log.cc:1196] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/a635ab5e12da4eacada29f98a92270b5
I20260811 10:16:14.271423 10599 ts_tablet_manager.cc:1939] T a635ab5e12da4eacada29f98a92270b5 P 7b949e362bb842caa0bbc949d7cc981b: Deleting consensus metadata
I20260811 10:16:14.271999 9658 catalog_manager.cc:4841] TS 7b949e362bb842caa0bbc949d7cc981b (127.8.238.3:45799): tablet a635ab5e12da4eacada29f98a92270b5 (table test-table [id=6c32317af2b24382aed5c8efce9fa07c]) successfully deleted
W20260811 10:16:14.276202 9654 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:39741 (error 108)
I20260811 10:16:14.276504 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10043
I20260811 10:16:14.279604 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.276675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:39741 (local address 127.0.0.1:48968)
0811 10:16:14.279497 (+ 2822us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.279558 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:39741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2789}
W20260811 10:16:14.279798 9654 catalog_manager.cc:4584] TS becd03cbb32341a58b944f551a5a76b0 (127.8.238.1:39741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a635ab5e12da4eacada29f98a92270b5: Network error: Client connection negotiation failed: client connection to 127.8.238.1:39741: connect: Connection refused (error 111)
I20260811 10:16:14.294656 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10308
I20260811 10:16:14.311524 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 9628
2026-08-11T10:16:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2570 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T10:16:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:14Z Disabled control of system clock
I20260811 10:16:14.355064 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46549
--webserver_interface=127.8.238.62
--webserver_port=0
--builtin_ntp_servers=127.8.238.20:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:46549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:14.449724 10611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:14.450016 10611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:14.450042 10611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:14.451730 10611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:14.451877 10611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:14.451891 10611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:14.451906 10611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:14.451918 10611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:14.453506 10611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:46549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:14.453763 10611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:14.454064 10611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:14.459506 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.459728 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.459535 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:14.459895 10611 server_base.cc:1034] running on GCE node
I20260811 10:16:14.460247 10611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:14.460543 10611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:14.461709 10611 hybrid_clock.cc:648] HybridClock initialized: now 1786443374461681 us; error 43 us; skew 500 ppm
I20260811 10:16:14.463218 10611 webserver.cc:460] Webserver started at http://127.8.238.62:39513/ using document root <none> and password file <none>
I20260811 10:16:14.463595 10611 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:14.463649 10611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:14.463842 10611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:14.465039 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "9ad6c1a3ec384e339b05abf032791e33"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a4fb02be2f7468c98d4fb577b42cb3fe"
server_key_iv: "e17c8bf28ddfede938339b48907932a1"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.465433 10611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "9ad6c1a3ec384e339b05abf032791e33"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a4fb02be2f7468c98d4fb577b42cb3fe"
server_key_iv: "e17c8bf28ddfede938339b48907932a1"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.466990 10611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 10:16:14.468050 10633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.468266 10611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:14.468380 10611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "9ad6c1a3ec384e339b05abf032791e33"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a4fb02be2f7468c98d4fb577b42cb3fe"
server_key_iv: "e17c8bf28ddfede938339b48907932a1"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.468466 10611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:14.480898 10611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:14.481292 10611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:14.481451 10611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:14.486409 10611 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:46549
I20260811 10:16:14.486449 10719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:46549 every 8 connection(s)
I20260811 10:16:14.486855 10611 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:14.488261 10720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:14.490959 10720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: Bootstrap starting.
I20260811 10:16:14.491578 10720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:14.491930 10720 log.cc:824] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:14.491942 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10611
I20260811 10:16:14.492312 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:14.492676 9144 external_mini_cluster.cc:1428] Setting key 8ed12894055e42e3a7659f5d9e0699d4
I20260811 10:16:14.492743 10720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: No bootstrap required, opened a new log
I20260811 10:16:14.494818 10720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } }
I20260811 10:16:14.495007 10720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:14.495059 10720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ad6c1a3ec384e339b05abf032791e33, State: Initialized, Role: FOLLOWER
I20260811 10:16:14.495226 10720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } }
I20260811 10:16:14.495328 10720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:14.495373 10720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:14.495414 10720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:14.496173 10720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } }
I20260811 10:16:14.496309 10720 leader_election.cc:302] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ad6c1a3ec384e339b05abf032791e33; no voters:
I20260811 10:16:14.496599 10720 leader_election.cc:288] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:14.496716 10731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:14.496944 10731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [term 1 LEADER]: Becoming Leader. State: Replica: 9ad6c1a3ec384e339b05abf032791e33, State: Running, Role: LEADER
I20260811 10:16:14.496943 10720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:14.497104 10731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } }
I20260811 10:16:14.497645 10732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ad6c1a3ec384e339b05abf032791e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } } }
I20260811 10:16:14.497653 10731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ad6c1a3ec384e339b05abf032791e33. Latest consensus state: current_term: 1 leader_uuid: "9ad6c1a3ec384e339b05abf032791e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad6c1a3ec384e339b05abf032791e33" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46549 } } }
I20260811 10:16:14.497789 10732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:14.497805 10731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:14.498389 10747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:14.499094 10747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:14.501856 10747 catalog_manager.cc:1347] Generated new cluster ID: 42f09b5d746b4f428c9e910ad0bfa7ea
I20260811 10:16:14.501938 10747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:14.502568 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.493541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.8.238.62:46549)
0811 10:16:14.493842 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.493852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.493886 (+ 34us) server_negotiation.cc:407] Connection header received
0811 10:16:14.494844 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.494855 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.494950 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.495255 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.495802 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.498196 (+ 2394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.499469 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.500170 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.500307 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.501560 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.501565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.501566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:14.501882 (+ 316us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.501942 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:14.501962 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.502054 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.502220 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.502339 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":125,"threads_started":1}
I20260811 10:16:14.533717 10747 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:14.534339 10747 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:14.541273 10747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: Generated new TSK 0
I20260811 10:16:14.541520 10747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:14.552409 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.1:0
--local_ip_for_outbound_sockets=127.8.238.1
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--builtin_ntp_servers=127.8.238.20:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:14.647488 10762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:14.647855 10762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:14.647904 10762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:14.649364 10762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:14.649408 10762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:14.649441 10762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.1
I20260811 10:16:14.651002 10762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.238.1
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.1
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:14.651268 10762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:14.651590 10762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:14.657903 10775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.657989 10769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.657905 10773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:14.658885 10762 server_base.cc:1034] running on GCE node
I20260811 10:16:14.659194 10762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:14.659567 10762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:14.660809 10762 hybrid_clock.cc:648] HybridClock initialized: now 1786443374660787 us; error 50 us; skew 500 ppm
I20260811 10:16:14.662580 10762 webserver.cc:460] Webserver started at http://127.8.238.1:37925/ using document root <none> and password file <none>
I20260811 10:16:14.662935 10762 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:14.663020 10762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:14.663194 10762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:14.664492 10762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data/instance:
uuid: "0013c90b37d64d0b83734ab075b0e93a"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "85d71ee2387050ac1f87e570aff2de67"
server_key_iv: "be3e3022747a13858a07df0233247c35"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.664896 10762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance:
uuid: "0013c90b37d64d0b83734ab075b0e93a"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "85d71ee2387050ac1f87e570aff2de67"
server_key_iv: "be3e3022747a13858a07df0233247c35"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.666543 10762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:14.667641 10783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.668011 10762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:14.668154 10762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal
uuid: "0013c90b37d64d0b83734ab075b0e93a"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "85d71ee2387050ac1f87e570aff2de67"
server_key_iv: "be3e3022747a13858a07df0233247c35"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.668246 10762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:14.681290 10762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:14.681738 10762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:14.681926 10762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:14.682214 10762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:14.682610 10762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:14.682667 10762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.682829 10762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:14.682888 10762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.690224 10762 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.1:35509
I20260811 10:16:14.690449 10982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.1:35509 every 8 connection(s)
I20260811 10:16:14.690737 10762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/data/info.pb
I20260811 10:16:14.698174 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.692044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:46259 (local address 127.8.238.62:46549)
0811 10:16:14.692154 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.692157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.692405 (+ 248us) server_negotiation.cc:407] Connection header received
0811 10:16:14.692994 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.692995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.693010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.693076 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.693977 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.694537 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.695846 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.696150 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.696268 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.697682 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.697683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.697684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:14.697690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.697715 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:14.697720 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.697803 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.698051 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.698090 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 10:16:14.698557 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.691977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46549 (local address 127.8.238.1:46259)
0811 10:16:14.692262 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.692301 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.692871 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.693211 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.693215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.693226 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:14.693850 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.693855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.694640 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.694641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.695592 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.695595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.696327 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.696329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.696517 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.697487 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:14.697488 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:14.697527 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:14.697564 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:14.697892 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:14.697901 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:14.697905 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.698000 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.698160 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":127,"threads_started":1}
I20260811 10:16:14.699179 10983 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46549
I20260811 10:16:14.699332 10983 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:14.699590 10983 heartbeater.cc:511] Master 127.8.238.62:46549 requested a full tablet report, sending...
I20260811 10:16:14.700596 10658 ts_manager.cc:194] Registered new tserver with Master: 0013c90b37d64d0b83734ab075b0e93a (127.8.238.1:35509)
I20260811 10:16:14.700666 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10762
I20260811 10:16:14.700763 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-0/wal/instance
I20260811 10:16:14.700995 9144 external_mini_cluster.cc:1428] Setting key affd34c8125a7a8635adcf5a85d8f44d
I20260811 10:16:14.701797 10658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.1:46259
I20260811 10:16:14.702633 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.2:0
--local_ip_for_outbound_sockets=127.8.238.2
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--builtin_ntp_servers=127.8.238.20:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:14.792616 10987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:14.792879 10987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:14.792905 10987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:14.794204 10987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:14.794265 10987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:14.794296 10987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.2
I20260811 10:16:14.795871 10987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.238.2
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.2
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:14.796168 10987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:14.796602 10987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:14.802732 11001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.802786 10993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.803399 10997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:14.804081 10987 server_base.cc:1034] running on GCE node
I20260811 10:16:14.804394 10987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:14.804693 10987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:14.805899 10987 hybrid_clock.cc:648] HybridClock initialized: now 1786443374805883 us; error 68 us; skew 500 ppm
I20260811 10:16:14.807564 10987 webserver.cc:460] Webserver started at http://127.8.238.2:46815/ using document root <none> and password file <none>
I20260811 10:16:14.807928 10987 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:14.808007 10987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:14.808183 10987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:14.809470 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data/instance:
uuid: "6d709d87d7f047b2975058d73ec2f207"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a1000425d71cd8e70d1f194e22c407ea"
server_key_iv: "096d8b5708fd1644737ed85db919c50e"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.810158 10987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance:
uuid: "6d709d87d7f047b2975058d73ec2f207"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a1000425d71cd8e70d1f194e22c407ea"
server_key_iv: "096d8b5708fd1644737ed85db919c50e"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.812258 10987 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 10:16:14.813517 11014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.813833 10987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:14.814029 10987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal
uuid: "6d709d87d7f047b2975058d73ec2f207"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "a1000425d71cd8e70d1f194e22c407ea"
server_key_iv: "096d8b5708fd1644737ed85db919c50e"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.814158 10987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:14.834416 10987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:14.834846 10987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:14.835029 10987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:14.835320 10987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:14.835774 10987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:14.835841 10987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.835878 10987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:14.835906 10987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.842479 10987 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.2:36127
I20260811 10:16:14.842545 11234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.2:36127 every 8 connection(s)
I20260811 10:16:14.842952 10987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/data/info.pb
I20260811 10:16:14.848932 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.843882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.2:42145 (local address 127.8.238.62:46549)
0811 10:16:14.843965 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.843967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.844245 (+ 278us) server_negotiation.cc:407] Connection header received
0811 10:16:14.844831 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.844833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.844847 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.844909 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.845666 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.846203 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.847269 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.847489 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.847567 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.848601 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.848602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.848603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:14.848610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.848629 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:14.848633 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.848694 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.848798 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.848835 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:16:14.849226 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.843868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46549 (local address 127.8.238.2:42145)
0811 10:16:14.844109 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.844145 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.844710 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.844994 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.844998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.845007 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:14.845563 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.845568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.846316 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.846318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.847164 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.847167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.847596 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.847597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.847765 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.848423 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:14.848423 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:14.848458 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:14.848486 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:14.848697 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:14.848705 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:14.848708 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.848800 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.848918 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260811 10:16:14.849848 11236 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46549
I20260811 10:16:14.849997 11236 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:14.850262 11236 heartbeater.cc:511] Master 127.8.238.62:46549 requested a full tablet report, sending...
I20260811 10:16:14.850261 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 10987
I20260811 10:16:14.850361 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal/instance
I20260811 10:16:14.850595 9144 external_mini_cluster.cc:1428] Setting key 8b2a2e0ffd36f2cd2735336408ee2dc0
I20260811 10:16:14.850778 10658 ts_manager.cc:194] Registered new tserver with Master: 6d709d87d7f047b2975058d73ec2f207 (127.8.238.2:36127)
I20260811 10:16:14.851300 10658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.2:42145
I20260811 10:16:14.851974 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.238.3:0
--local_ip_for_outbound_sockets=127.8.238.3
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--builtin_ntp_servers=127.8.238.20:35429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 10:16:14.948112 11239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:14.948370 11239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:14.948395 11239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:14.949700 11239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:14.949752 11239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:14.949781 11239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.238.3
I20260811 10:16:14.951278 11239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:35429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.238.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.238.3
--webserver_port=0
--tserver_master_addrs=127.8.238.62:46549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.238.3
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:14.951591 11239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:14.952015 11239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:14.957998 11246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.958094 11250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:14.958101 11259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:14.958405 11239 server_base.cc:1034] running on GCE node
I20260811 10:16:14.958715 11239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:14.959044 11239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:14.960242 11239 hybrid_clock.cc:648] HybridClock initialized: now 1786443374960214 us; error 53 us; skew 500 ppm
I20260811 10:16:14.961881 11239 webserver.cc:460] Webserver started at http://127.8.238.3:44099/ using document root <none> and password file <none>
I20260811 10:16:14.962256 11239 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:14.962314 11239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:14.962469 11239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:14.963696 11239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data/instance:
uuid: "a84788d5f56d4bf6888aa3f2151114da"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "4f182bab789f9306ed879623001dec84"
server_key_iv: "4d49b499f4bed2a9c04dcbdd634a52a2"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.964196 11239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance:
uuid: "a84788d5f56d4bf6888aa3f2151114da"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "4f182bab789f9306ed879623001dec84"
server_key_iv: "4d49b499f4bed2a9c04dcbdd634a52a2"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.965763 11239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:14.966739 11270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.967132 11239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:14.967265 11239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal
uuid: "a84788d5f56d4bf6888aa3f2151114da"
format_stamp: "Formatted at 2026-08-11 10:16:14 on dist-test-slave-7f01"
server_key: "4f182bab789f9306ed879623001dec84"
server_key_iv: "4d49b499f4bed2a9c04dcbdd634a52a2"
server_key_version: "encryptionkey@0"
I20260811 10:16:14.967355 11239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:14.978902 11239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:14.979442 11239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:14.979645 11239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:14.980067 11239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 10:16:14.980544 11239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 10:16:14.980607 11239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.980659 11239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 10:16:14.980851 11239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:14.987866 11239 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.3:46861
I20260811 10:16:14.987918 11472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.3:46861 every 8 connection(s)
I20260811 10:16:14.988370 11239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/data/info.pb
I20260811 10:16:14.990257 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11239
I20260811 10:16:14.990381 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal/instance
I20260811 10:16:14.990612 9144 external_mini_cluster.cc:1428] Setting key 6532018152b5b92cc7adbc092a37c6ae
I20260811 10:16:14.995371 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.989382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.3:53631 (local address 127.8.238.62:46549)
0811 10:16:14.989534 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:14.989537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:14.989741 (+ 204us) server_negotiation.cc:407] Connection header received
0811 10:16:14.990641 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:14.990642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:14.990686 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:14.990761 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:14.991861 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.992467 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.993544 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.993737 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.993807 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.994873 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:14.994876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:14.994876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:14.994885 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:14.994917 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:14.994924 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:14.995181 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:14.995256 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 10:16:14.995290 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 10:16:14.995622 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:14.989233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:46549 (local address 127.8.238.3:53631)
0811 10:16:14.989599 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:14.989647 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:14.990496 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:14.990911 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:14.990914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:14.990926 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:14.991767 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.991776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.992577 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.992578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.993440 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:14.993443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:14.993836 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:14.993837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:14.993970 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:14.994635 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 10:16:14.994636 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:14.994670 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 10:16:14.994699 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:14.995183 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:14.995189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:14.995192 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:14.995255 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 10:16:14.995363 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260811 10:16:14.996150 11473 heartbeater.cc:347] Connected to a master server at 127.8.238.62:46549
I20260811 10:16:14.996258 11473 heartbeater.cc:464] Registering TS with master...
I20260811 10:16:14.996436 11473 heartbeater.cc:511] Master 127.8.238.62:46549 requested a full tablet report, sending...
I20260811 10:16:14.996987 10658 ts_manager.cc:194] Registered new tserver with Master: a84788d5f56d4bf6888aa3f2151114da (127.8.238.3:46861)
I20260811 10:16:14.997365 10658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.238.3:53631
I20260811 10:16:15.003118 9144 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 10:16:15.009392 10727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.005437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.8.238.62:46549)
0811 10:16:15.005543 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.005545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.005682 (+ 137us) server_negotiation.cc:407] Connection header received
0811 10:16:15.005713 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.005714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.005731 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.005798 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:15.006210 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.006858 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.007937 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.008195 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.008280 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.009080 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:15.009082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:15.009082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:15.009087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:15.009104 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:15.009108 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:15.009161 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.009286 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.009316 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 10:16:15.015445 10658 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 10:16:15.017861 10658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 10:16:15.025804 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.8.238.2:36127)
0811 10:16:15.021873 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.021879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.021891 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:15.021937 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.021940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.021957 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.022257 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:15.022333 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023113 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.024682 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.025361 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.025452 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.025561 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.025659 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.025701 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":109,"threads_started":1}
I20260811 10:16:15.025792 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:36127 (local address 127.0.0.1:60268)
0811 10:16:15.021590 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.021606 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:15.021690 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:15.022072 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:15.022074 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:15.022077 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:15.022174 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.022177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023217 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.023218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.024582 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.024585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.025466 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.025467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.025551 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.025566 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 10:16:15.025644 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:16:15.025702 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":121,"threads_started":1}
I20260811 10:16:15.025972 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:46861 (local address 127.0.0.1:40582)
0811 10:16:15.021603 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.021613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:15.021687 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:15.022285 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:15.022285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:15.022287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:15.022326 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.022327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023631 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.023632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.024712 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.024714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.025724 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.025725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.025789 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.025794 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 10:16:15.025842 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 10:16:15.025895 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":59,"threads_started":1}
I20260811 10:16:15.026284 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:35509 (local address 127.0.0.1:56572)
0811 10:16:15.021530 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.021578 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:15.021686 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:15.022037 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:15.022041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:15.022044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:15.022139 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.022143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023883 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.023885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.024890 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.024893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.026038 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.026039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.026118 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.026121 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 10:16:15.026172 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 10:16:15.026220 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":119,"threads_started":1}
I20260811 10:16:15.026383 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.8.238.3:46861)
0811 10:16:15.022113 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.022118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.022130 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:15.022175 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.022177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.022195 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.022288 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:15.022412 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023530 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.024831 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.025609 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.026104 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.026207 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.026236 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.026283 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":140,"threads_started":1}
I20260811 10:16:15.026453 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.021281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.8.238.1:35509)
0811 10:16:15.021492 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.021498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.021657 (+ 159us) server_negotiation.cc:407] Connection header received
0811 10:16:15.021810 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.021812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.021830 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.021915 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:15.022242 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.023523 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.025024 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.025843 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.025909 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.025998 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.026206 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.026260 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":121,"threads_started":1}
I20260811 10:16:15.027158 10868 tablet_service.cc:1467] Processing CreateTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 (DEFAULT_TABLE table=test-table [id=4cffce2521e14c3c9306653369a09bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:15.027161 11352 tablet_service.cc:1467] Processing CreateTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 (DEFAULT_TABLE table=test-table [id=4cffce2521e14c3c9306653369a09bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:15.027161 11107 tablet_service.cc:1467] Processing CreateTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 (DEFAULT_TABLE table=test-table [id=4cffce2521e14c3c9306653369a09bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 10:16:15.027571 10868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab1ce7d224b34e32872abcb27f65e7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:15.027572 11107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab1ce7d224b34e32872abcb27f65e7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:15.027573 11352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab1ce7d224b34e32872abcb27f65e7a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:15.030689 11504 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: Bootstrap starting.
I20260811 10:16:15.031119 11505 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Bootstrap starting.
I20260811 10:16:15.031327 11508 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Bootstrap starting.
I20260811 10:16:15.031791 11504 tablet_bootstrap.cc:654] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:15.031791 11505 tablet_bootstrap.cc:654] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:15.032022 11508 tablet_bootstrap.cc:654] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:15.032174 11505 log.cc:824] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:15.032174 11504 log.cc:824] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:15.032395 11508 log.cc:824] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:15.033089 11505 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: No bootstrap required, opened a new log
I20260811 10:16:15.033125 11504 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: No bootstrap required, opened a new log
I20260811 10:16:15.033154 11508 tablet_bootstrap.cc:492] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: No bootstrap required, opened a new log
I20260811 10:16:15.033174 11505 ts_tablet_manager.cc:1397] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 10:16:15.033190 11504 ts_tablet_manager.cc:1397] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 10:16:15.033226 11508 ts_tablet_manager.cc:1397] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 10:16:15.035091 11505 raft_consensus.cc:348] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.035276 11505 raft_consensus.cc:374] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:15.035331 11505 raft_consensus.cc:729] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d709d87d7f047b2975058d73ec2f207, State: Initialized, Role: FOLLOWER
I20260811 10:16:15.035461 11505 consensus_queue.cc:260] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.035804 11505 ts_tablet_manager.cc:1428] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 10:16:15.035959 11236 heartbeater.cc:503] Master 127.8.238.62:46549 was elected leader, sending a full tablet report...
I20260811 10:16:15.035950 11504 raft_consensus.cc:348] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.036135 11504 raft_consensus.cc:374] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:15.036172 11504 raft_consensus.cc:729] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0013c90b37d64d0b83734ab075b0e93a, State: Initialized, Role: FOLLOWER
I20260811 10:16:15.036289 11504 consensus_queue.cc:260] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.036412 11508 raft_consensus.cc:348] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.036587 11504 ts_tablet_manager.cc:1428] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 10:16:15.036581 11508 raft_consensus.cc:374] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:15.036633 11508 raft_consensus.cc:729] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84788d5f56d4bf6888aa3f2151114da, State: Initialized, Role: FOLLOWER
I20260811 10:16:15.036757 11508 consensus_queue.cc:260] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.036868 10983 heartbeater.cc:503] Master 127.8.238.62:46549 was elected leader, sending a full tablet report...
I20260811 10:16:15.037245 11473 heartbeater.cc:503] Master 127.8.238.62:46549 was elected leader, sending a full tablet report...
I20260811 10:16:15.037225 11508 ts_tablet_manager.cc:1428] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260811 10:16:15.093739 11237 tablet.cc:2369] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:15.108166 11516 raft_consensus.cc:482] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 10:16:15.108399 11516 raft_consensus.cc:504] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.108846 11516 leader_election.cc:288] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d709d87d7f047b2975058d73ec2f207 (127.8.238.2:36127), a84788d5f56d4bf6888aa3f2151114da (127.8.238.3:46861)
I20260811 10:16:15.113536 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.109228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:40393 (local address 127.8.238.2:36127)
0811 10:16:15.109296 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.109299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.109313 (+ 14us) server_negotiation.cc:407] Connection header received
0811 10:16:15.109396 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.109397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.109412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.109479 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:15.109727 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.110479 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.112464 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.113060 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.113153 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.113166 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.113422 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.113462 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:16:15.113536 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.109436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.238.1:33557 (local address 127.8.238.3:46861)
0811 10:16:15.109487 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.109489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.109591 (+ 102us) server_negotiation.cc:407] Connection header received
0811 10:16:15.109731 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.109733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.109744 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.109827 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 10:16:15.110048 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.110694 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.112463 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.113068 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.113153 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.113166 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.113414 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.113459 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:16:15.113544 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.109116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.2:36127 (local address 127.8.238.1:40393)
0811 10:16:15.109194 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.109209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:15.109286 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:15.109523 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:15.109524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:15.109526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:15.109621 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.109624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.110656 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.110659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.112308 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.112312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.113212 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.113213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.113309 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.113317 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:16:15.113401 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 10:16:15.113444 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":42}
I20260811 10:16:15.113551 11520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.109270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.3:46861 (local address 127.8.238.1:33557)
0811 10:16:15.109508 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.109519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:15.109583 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:15.109876 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:15.109877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:15.109879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 10:16:15.109962 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.109963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.110813 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.110814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.112308 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:15.112312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.113225 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.113226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:15.113309 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.113317 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 10:16:15.113389 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 10:16:15.113440 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":39,"thread_start_us":174,"threads_started":1}
I20260811 10:16:15.113983 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab1ce7d224b34e32872abcb27f65e7a2" candidate_uuid: "0013c90b37d64d0b83734ab075b0e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84788d5f56d4bf6888aa3f2151114da" is_pre_election: true
I20260811 10:16:15.114014 11141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab1ce7d224b34e32872abcb27f65e7a2" candidate_uuid: "0013c90b37d64d0b83734ab075b0e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d709d87d7f047b2975058d73ec2f207" is_pre_election: true
I20260811 10:16:15.114190 11384 raft_consensus.cc:2454] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0013c90b37d64d0b83734ab075b0e93a in term 0.
I20260811 10:16:15.114208 11141 raft_consensus.cc:2454] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0013c90b37d64d0b83734ab075b0e93a in term 0.
I20260811 10:16:15.114454 10784 leader_election.cc:302] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0013c90b37d64d0b83734ab075b0e93a, a84788d5f56d4bf6888aa3f2151114da; no voters:
I20260811 10:16:15.114719 11516 raft_consensus.cc:2781] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 10:16:15.114883 11516 raft_consensus.cc:482] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 10:16:15.114908 11516 raft_consensus.cc:3037] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:15.115998 11516 raft_consensus.cc:504] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.116147 11516 leader_election.cc:288] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [CANDIDATE]: Term 1 election: Requested vote from peers 6d709d87d7f047b2975058d73ec2f207 (127.8.238.2:36127), a84788d5f56d4bf6888aa3f2151114da (127.8.238.3:46861)
I20260811 10:16:15.116533 11384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab1ce7d224b34e32872abcb27f65e7a2" candidate_uuid: "0013c90b37d64d0b83734ab075b0e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a84788d5f56d4bf6888aa3f2151114da"
I20260811 10:16:15.116616 11384 raft_consensus.cc:3037] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:15.116524 11141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab1ce7d224b34e32872abcb27f65e7a2" candidate_uuid: "0013c90b37d64d0b83734ab075b0e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d709d87d7f047b2975058d73ec2f207"
I20260811 10:16:15.116691 11141 raft_consensus.cc:3037] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:15.117641 11384 raft_consensus.cc:2454] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0013c90b37d64d0b83734ab075b0e93a in term 1.
I20260811 10:16:15.117893 10784 leader_election.cc:302] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0013c90b37d64d0b83734ab075b0e93a, a84788d5f56d4bf6888aa3f2151114da; no voters:
I20260811 10:16:15.117974 11141 raft_consensus.cc:2454] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0013c90b37d64d0b83734ab075b0e93a in term 1.
I20260811 10:16:15.118077 11516 raft_consensus.cc:2781] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:15.118324 11516 raft_consensus.cc:686] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [term 1 LEADER]: Becoming Leader. State: Replica: 0013c90b37d64d0b83734ab075b0e93a, State: Running, Role: LEADER
I20260811 10:16:15.118501 11516 consensus_queue.cc:237] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } }
I20260811 10:16:15.119796 10657 catalog_manager.cc:5495] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0013c90b37d64d0b83734ab075b0e93a (127.8.238.1). New cstate: current_term: 1 leader_uuid: "0013c90b37d64d0b83734ab075b0e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0013c90b37d64d0b83734ab075b0e93a" member_type: VOTER last_known_addr { host: "127.8.238.1" port: 35509 } health_report { overall_health: HEALTHY } } }
I20260811 10:16:15.149470 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.146795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.8.238.1:35509)
0811 10:16:15.146872 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:15.146874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:15.146881 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:16:15.146922 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:15.146923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:15.146953 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:15.147042 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:15.147210 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.147711 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.148574 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:15.148777 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:15.148855 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:15.149069 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:15.149186 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:15.149292 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:15.149312 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 10:16:15.149340 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 10:16:15.151571 11384 raft_consensus.cc:1261] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 1 FOLLOWER]: Refusing update from remote peer 0013c90b37d64d0b83734ab075b0e93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:15.151615 11141 raft_consensus.cc:1261] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 1 FOLLOWER]: Refusing update from remote peer 0013c90b37d64d0b83734ab075b0e93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:15.152060 11516 consensus_queue.cc:1035] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d709d87d7f047b2975058d73ec2f207" member_type: VOTER last_known_addr { host: "127.8.238.2" port: 36127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:15.152232 11521 consensus_queue.cc:1035] T ab1ce7d224b34e32872abcb27f65e7a2 P 0013c90b37d64d0b83734ab075b0e93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a84788d5f56d4bf6888aa3f2151114da" member_type: VOTER last_known_addr { host: "127.8.238.3" port: 46861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:15.155465 11528 mvcc.cc:205] Tried to move back new op lower bound from 7317272064617598976 to 7317272064486297600. Current Snapshot: MvccSnapshot[applied={T|T < 7317272064617598976}]
I20260811 10:16:15.155483 11526 mvcc.cc:205] Tried to move back new op lower bound from 7317272064617598976 to 7317272064486297600. Current Snapshot: MvccSnapshot[applied={T|T < 7317272064617598976 or (T in {7317272064617598976})}]
I20260811 10:16:15.158769 10658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:15.158959 10658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 10:16:15.160307 10658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ad6c1a3ec384e339b05abf032791e33: Sending DeleteTablet for 3 replicas of tablet ab1ce7d224b34e32872abcb27f65e7a2
I20260811 10:16:15.160785 10868 tablet_service.cc:1514] Processing DeleteTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:15 UTC) from {username='slave'} at 127.0.0.1:56572
I20260811 10:16:15.160929 11107 tablet_service.cc:1514] Processing DeleteTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:15 UTC) from {username='slave'} at 127.0.0.1:60268
I20260811 10:16:15.160959 11352 tablet_service.cc:1514] Processing DeleteTablet for tablet ab1ce7d224b34e32872abcb27f65e7a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 10:16:15 UTC) from {username='slave'} at 127.0.0.1:40582
I20260811 10:16:15.161170 11547 tablet_replica.cc:330] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: stopping tablet replica
I20260811 10:16:15.161298 11547 raft_consensus.cc:2229] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:15.161429 11547 raft_consensus.cc:2258] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:15.161417 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10762
I20260811 10:16:15.161598 11548 tablet_replica.cc:330] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: stopping tablet replica
I20260811 10:16:15.161727 11548 raft_consensus.cc:2229] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 10:16:15.161870 11548 raft_consensus.cc:2258] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 10:16:15.165419 11548 ts_tablet_manager.cc:1905] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:15.167725 11548 ts_tablet_manager.cc:1918] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:15.167903 11548 log.cc:1196] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-2/wal/wals/ab1ce7d224b34e32872abcb27f65e7a2
I20260811 10:16:15.168503 11548 ts_tablet_manager.cc:1939] T ab1ce7d224b34e32872abcb27f65e7a2 P a84788d5f56d4bf6888aa3f2151114da: Deleting consensus metadata
I20260811 10:16:15.168982 10634 catalog_manager.cc:4841] TS a84788d5f56d4bf6888aa3f2151114da (127.8.238.3:46861): tablet ab1ce7d224b34e32872abcb27f65e7a2 (table test-table [id=4cffce2521e14c3c9306653369a09bcd]) successfully deleted
I20260811 10:16:15.176415 11547 ts_tablet_manager.cc:1905] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 10:16:15.179008 11547 ts_tablet_manager.cc:1918] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 10:16:15.179136 11547 log.cc:1196] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Deleting WAL directory at /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786443124069174-9144-0/minicluster-data/ts-1/wal/wals/ab1ce7d224b34e32872abcb27f65e7a2
I20260811 10:16:15.179425 11547 ts_tablet_manager.cc:1939] T ab1ce7d224b34e32872abcb27f65e7a2 P 6d709d87d7f047b2975058d73ec2f207: Deleting consensus metadata
I20260811 10:16:15.180557 10634 catalog_manager.cc:4841] TS 6d709d87d7f047b2975058d73ec2f207 (127.8.238.2:36127): tablet ab1ce7d224b34e32872abcb27f65e7a2 (table test-table [id=4cffce2521e14c3c9306653369a09bcd]) successfully deleted
W20260811 10:16:15.180716 10637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.238.1:35509 (error 108)
I20260811 10:16:15.180883 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10987
I20260811 10:16:15.188201 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:15.187988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.1:35509 (local address 127.0.0.1:56584)
0811 10:16:15.188072 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:15.188141 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.1:35509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 10:16:15.188375 10637 catalog_manager.cc:4584] TS 0013c90b37d64d0b83734ab075b0e93a (127.8.238.1:35509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab1ce7d224b34e32872abcb27f65e7a2: Network error: Client connection negotiation failed: client connection to 127.8.238.1:35509: connect: Connection refused (error 111)
I20260811 10:16:15.195063 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11239
I20260811 10:16:15.214005 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 10611
2026-08-11T10:16:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (898 ms)
[----------] 6 tests from EncryptionPolicyTest (10289 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:15 dist-test-slave-7f01 krb5kdc[11558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:15 dist-test-slave-7f01 krb5kdc[11558](info): set up 2 sockets
Aug 11 10:16:15 dist-test-slave-7f01 krb5kdc[11558](info): commencing operation
krb5kdc: starting...
W20260811 10:16:16.537568 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:16.697379 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:16.788365 11578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:16.788671 11578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:16.788717 11578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:16.790020 11578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:16.790105 11578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:16.790131 11578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:16.790162 11578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:16.790192 11578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:16.791890 11578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:16.792210 11578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:16.792476 11578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:16.798458 11590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:16.798478 11599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:16.798527 11593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:16.798731 11578 server_base.cc:1034] running on GCE node
I20260811 10:16:16.799227 11578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:16.799654 11578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:16.801046 11578 hybrid_clock.cc:648] HybridClock initialized: now 1786443376801017 us; error 49 us; skew 500 ppm
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:16.804979 11578 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:16.805730 11578 webserver.cc:460] Webserver started at http://127.0.0.1:38249/ using document root <none> and password file <none>
I20260811 10:16:16.806080 11578 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:16.806167 11578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:16.806362 11578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:16.807960 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "354e8ae1647f4f21a168ea006d9704b0"
format_stamp: "Formatted at 2026-08-11 10:16:16 on dist-test-slave-7f01"
server_key: "8d39fa7d9e40c6a7f9e785904fc82d52"
server_key_iv: "77c9a86bee2cb4f188bcf1b0b684995c"
server_key_version: "encryptionkey@0"
I20260811 10:16:16.808447 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "354e8ae1647f4f21a168ea006d9704b0"
format_stamp: "Formatted at 2026-08-11 10:16:16 on dist-test-slave-7f01"
server_key: "8d39fa7d9e40c6a7f9e785904fc82d52"
server_key_iv: "77c9a86bee2cb4f188bcf1b0b684995c"
server_key_version: "encryptionkey@0"
I20260811 10:16:16.810444 11578 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 10:16:16.811705 11608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:16.812201 11578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:16:16.812342 11578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "354e8ae1647f4f21a168ea006d9704b0"
format_stamp: "Formatted at 2026-08-11 10:16:16 on dist-test-slave-7f01"
server_key: "8d39fa7d9e40c6a7f9e785904fc82d52"
server_key_iv: "77c9a86bee2cb4f188bcf1b0b684995c"
server_key_version: "encryptionkey@0"
I20260811 10:16:16.812449 11578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:16.822543 11578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:16.825146 11578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:16.825440 11578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:16.830467 11578 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32869
I20260811 10:16:16.830520 11694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32869 every 8 connection(s)
I20260811 10:16:16.830930 11578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:16.832091 11696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:16.834575 11696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0: Bootstrap starting.
I20260811 10:16:16.835194 11696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:16.835472 11696 log.cc:824] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:16.836333 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11578
I20260811 10:16:16.836469 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:16.836678 11696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0: No bootstrap required, opened a new log
I20260811 10:16:16.836728 9144 external_mini_cluster.cc:1428] Setting key a713d057b46aec8dd3cdafba65e20778
I20260811 10:16:16.839293 11696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } }
I20260811 10:16:16.839556 11696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:16.839587 11696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354e8ae1647f4f21a168ea006d9704b0, State: Initialized, Role: FOLLOWER
I20260811 10:16:16.839855 11696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } }
I20260811 10:16:16.839962 11696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:16.839995 11696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:16.840095 11696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:16.841053 11696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } }
I20260811 10:16:16.841228 11696 leader_election.cc:302] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354e8ae1647f4f21a168ea006d9704b0; no voters:
I20260811 10:16:16.841579 11707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:16.841558 11696 leader_election.cc:288] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:16.841802 11707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [term 1 LEADER]: Becoming Leader. State: Replica: 354e8ae1647f4f21a168ea006d9704b0, State: Running, Role: LEADER
I20260811 10:16:16.841933 11707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } }
I20260811 10:16:16.842101 11696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:16.842515 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354e8ae1647f4f21a168ea006d9704b0. Latest consensus state: current_term: 1 leader_uuid: "354e8ae1647f4f21a168ea006d9704b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } } }
I20260811 10:16:16.842489 11707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354e8ae1647f4f21a168ea006d9704b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354e8ae1647f4f21a168ea006d9704b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32869 } } }
I20260811 10:16:16.842680 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:16.842685 11707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:16.843129 11723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:16.843804 11723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:16:16.845832 11723 catalog_manager.cc:1347] Generated new cluster ID: c4a6e9ca10a14768a5128f584a07ac88
I20260811 10:16:16.845899 11723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:16.849440 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:16.837383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56245 (local address 127.0.0.1:32869)
0811 10:16:16.837587 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:16.837597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:16.837625 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:16:16.838127 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:16.838134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:16.838183 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:16.838351 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:16.839054 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:16.840572 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:16.841445 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:16.842083 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:16.842199 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:16.845657 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:16.845662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:16.845663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:16.845677 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:16.847669 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:16.848176 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:16.848180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:16.848181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:16.848289 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:16.848526 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:16.848526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:16.848527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:16.848811 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:16.848998 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:16.849116 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 10:16:16.849233 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260811 10:16:16.850489 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:16.854821 11723 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:16.855587 11723 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:16.863385 11723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354e8ae1647f4f21a168ea006d9704b0: Generated new TSK 0
I20260811 10:16:16.863806 11723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:16.868527 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:16.852099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59533 (local address 127.0.0.1:32869)
0811 10:16:16.852143 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:16.852145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:16.854407 (+ 2262us) server_negotiation.cc:407] Connection header received
0811 10:16:16.854560 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:16.854562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:16.854575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:16.854659 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:16.860933 (+ 6274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:16.861605 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:16.862502 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:16.862838 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:16.862935 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:16.865070 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:16.865072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:16.865072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:16.865082 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:16.867013 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:16.867501 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:16.867502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:16.867503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:16.867566 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:16.867847 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:16.867848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:16.867848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:16.867996 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:16.868087 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:16.868300 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 10:16:16.868396 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:16:16.871162 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11578
W20260811 10:16:16.896090 11551 connection.cc:535] client connection to 127.0.0.1:32869 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32869): Connection reset by peer (error 104)
2026-08-11T10:16:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1668 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11757](info): set up 2 sockets
Aug 11 10:16:16 dist-test-slave-7f01 krb5kdc[11757](info): commencing operation
krb5kdc: starting...
W20260811 10:16:18.240742 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:18 dist-test-slave-7f01 krb5kdc[11757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:18.413379 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:18.516677 11777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:18.517012 11777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:18.517064 11777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:18.518553 11777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:18.518642 11777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:18.518667 11777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:18.518692 11777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:18.518723 11777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:18.520624 11777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:18.520996 11777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:18.521284 11777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:18.527050 11801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:18.527154 11798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:18.527096 11777 server_base.cc:1034] running on GCE node
W20260811 10:16:18.527055 11792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:18.528035 11777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:18.528469 11777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:18.529673 11777 hybrid_clock.cc:648] HybridClock initialized: now 1786443378529642 us; error 49 us; skew 500 ppm
Aug 11 10:16:18 dist-test-slave-7f01 krb5kdc[11757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443378, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:18.533159 11777 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:18.533809 11777 webserver.cc:460] Webserver started at http://127.0.0.1:39735/ using document root <none> and password file <none>
I20260811 10:16:18.534104 11777 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:18.534188 11777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:18.534363 11777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:18.535656 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "5eee1d8eeaab458e83506fc891c284cb"
format_stamp: "Formatted at 2026-08-11 10:16:18 on dist-test-slave-7f01"
server_key: "ec33bd1da48304672fd1388428848d8e"
server_key_iv: "eb858cc1abde2bba1cd616fa912fd4bd"
server_key_version: "encryptionkey@0"
I20260811 10:16:18.536164 11777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "5eee1d8eeaab458e83506fc891c284cb"
format_stamp: "Formatted at 2026-08-11 10:16:18 on dist-test-slave-7f01"
server_key: "ec33bd1da48304672fd1388428848d8e"
server_key_iv: "eb858cc1abde2bba1cd616fa912fd4bd"
server_key_version: "encryptionkey@0"
I20260811 10:16:18.537899 11777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:18.539180 11817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:18.539580 11777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:18.539806 11777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "5eee1d8eeaab458e83506fc891c284cb"
format_stamp: "Formatted at 2026-08-11 10:16:18 on dist-test-slave-7f01"
server_key: "ec33bd1da48304672fd1388428848d8e"
server_key_iv: "eb858cc1abde2bba1cd616fa912fd4bd"
server_key_version: "encryptionkey@0"
I20260811 10:16:18.539940 11777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:18.560714 11777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:18.564056 11777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:18.564250 11777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:18.569334 11777 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38161
I20260811 10:16:18.569375 11912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38161 every 8 connection(s)
I20260811 10:16:18.569783 11777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:18.571029 11913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:18.573199 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11777
I20260811 10:16:18.573381 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:18.573623 9144 external_mini_cluster.cc:1428] Setting key c61997378ea92e4d05fb12ae02aea7a4
I20260811 10:16:18.574230 11913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: Bootstrap starting.
I20260811 10:16:18.575192 11913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:18.575600 11913 log.cc:824] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:18.576508 11913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: No bootstrap required, opened a new log
I20260811 10:16:18.578711 11913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } }
I20260811 10:16:18.578909 11913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:18.578936 11913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eee1d8eeaab458e83506fc891c284cb, State: Initialized, Role: FOLLOWER
I20260811 10:16:18.579150 11913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } }
I20260811 10:16:18.579260 11913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:18.579355 11913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:18.579429 11913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:18.580354 11913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } }
I20260811 10:16:18.580523 11913 leader_election.cc:302] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eee1d8eeaab458e83506fc891c284cb; no voters:
I20260811 10:16:18.580785 11913 leader_election.cc:288] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:18.580829 11927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:18.581076 11927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [term 1 LEADER]: Becoming Leader. State: Replica: 5eee1d8eeaab458e83506fc891c284cb, State: Running, Role: LEADER
I20260811 10:16:18.581267 11913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:18.581599 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:18.574313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51473 (local address 127.0.0.1:38161)
0811 10:16:18.574569 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:18.574581 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:18.574619 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:16:18.575432 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:18.575446 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:18.575527 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:18.575727 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:18.576250 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:18.577472 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:18.578408 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:18.578964 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:18.579142 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:18.580253 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:18.580259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:18.580259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:18.580527 (+ 268us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:18.580689 (+ 162us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:18.580738 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:18.580909 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:18.581091 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 10:16:18.581136 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":122,"threads_started":1}
I20260811 10:16:18.581261 11927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } }
I20260811 10:16:18.582434 11930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eee1d8eeaab458e83506fc891c284cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } } }
I20260811 10:16:18.582710 11930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:18.582860 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:18.582636 11929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eee1d8eeaab458e83506fc891c284cb. Latest consensus state: current_term: 1 leader_uuid: "5eee1d8eeaab458e83506fc891c284cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eee1d8eeaab458e83506fc891c284cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38161 } } }
I20260811 10:16:18.582947 11929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:18.583956 11952 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:18.584035 11952 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:18.584158 11940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:18.584380 11940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:18.586373 11940 catalog_manager.cc:1347] Generated new cluster ID: 00054269085240bda5464804e8dd4361
I20260811 10:16:18.586443 11940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:18.588562 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:18.584602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35279 (local address 127.0.0.1:38161)
0811 10:16:18.584650 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:18.584652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:18.584850 (+ 198us) server_negotiation.cc:407] Connection header received
0811 10:16:18.584901 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:18.584903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:18.584916 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:18.584989 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:18.585160 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:18.585847 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:18.586975 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:18.587332 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:18.587412 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:18.588267 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:18.588269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:18.588269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:18.588275 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:18.588294 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:18.588299 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:18.588369 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:18.588460 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 10:16:18.588473 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:16:18.597662 11940 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:18.598501 11940 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:18.605250 11940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eee1d8eeaab458e83506fc891c284cb: Generated new TSK 0
I20260811 10:16:18.605535 11940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:18.608443 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11777
W20260811 10:16:18.629172 11754 connection.cc:535] client connection to 127.0.0.1:38161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38161): Connection reset by peer (error 104)
2026-08-11T10:16:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1732 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:18 dist-test-slave-7f01 krb5kdc[11968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:18 dist-test-slave-7f01 krb5kdc[11968](info): set up 2 sockets
Aug 11 10:16:18 dist-test-slave-7f01 krb5kdc[11968](info): commencing operation
krb5kdc: starting...
W20260811 10:16:19.968864 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:20 dist-test-slave-7f01 krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:20.124920 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:20.222103 11992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:20.222389 11992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:20.222436 11992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:20.223686 11992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:20.223785 11992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:20.223826 11992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:20.223858 11992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:20.223881 11992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:20.225590 11992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45733
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:20.225922 11992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:20.226168 11992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:20.231549 11999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:20.231695 11992 server_base.cc:1034] running on GCE node
W20260811 10:16:20.231575 12002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:20.231575 12006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:20.232242 11992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:20.232604 11992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:20.233824 11992 hybrid_clock.cc:648] HybridClock initialized: now 1786443380233804 us; error 73 us; skew 500 ppm
Aug 11 10:16:20 dist-test-slave-7f01 krb5kdc[11968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443380, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:20.241889 11992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:20.242551 11992 webserver.cc:460] Webserver started at http://127.0.0.1:34719/ using document root <none> and password file <none>
I20260811 10:16:20.242911 11992 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:20.242976 11992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:20.243104 11992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:20.244664 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "4d7fefdc2c374a0ba46e9c223fea9640"
format_stamp: "Formatted at 2026-08-11 10:16:20 on dist-test-slave-7f01"
server_key: "719cc966df8c013060f02ddf7ec41e1c"
server_key_iv: "e5e72a31334aa0ed7092a0b0fd648cbf"
server_key_version: "encryptionkey@0"
I20260811 10:16:20.245070 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "4d7fefdc2c374a0ba46e9c223fea9640"
format_stamp: "Formatted at 2026-08-11 10:16:20 on dist-test-slave-7f01"
server_key: "719cc966df8c013060f02ddf7ec41e1c"
server_key_iv: "e5e72a31334aa0ed7092a0b0fd648cbf"
server_key_version: "encryptionkey@0"
I20260811 10:16:20.246707 11992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 10:16:20.247905 12021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:20.248132 11992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 10:16:20.248229 11992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "4d7fefdc2c374a0ba46e9c223fea9640"
format_stamp: "Formatted at 2026-08-11 10:16:20 on dist-test-slave-7f01"
server_key: "719cc966df8c013060f02ddf7ec41e1c"
server_key_iv: "e5e72a31334aa0ed7092a0b0fd648cbf"
server_key_version: "encryptionkey@0"
I20260811 10:16:20.248340 11992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:20.252765 11992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:20.255599 11992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:20.255841 11992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:20.261096 11992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45733
I20260811 10:16:20.261185 12118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45733 every 8 connection(s)
I20260811 10:16:20.261610 11992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:20.262912 12119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:20.263016 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 11992
I20260811 10:16:20.263155 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:20.263427 9144 external_mini_cluster.cc:1428] Setting key 5bb6e34cf5a62b1a4ada07f554ee3436
I20260811 10:16:20.266670 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:20.264194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59177 (local address 127.0.0.1:45733)
0811 10:16:20.264453 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:20.264464 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:20.264503 (+ 39us) server_negotiation.cc:407] Connection header received
0811 10:16:20.265128 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:20.265139 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:20.265180 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:20.265366 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:20.266126 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:20.266131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:20.266131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:20.266136 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:20.266191 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:20.266207 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:20.266289 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:20.266410 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 10:16:20.266454 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260811 10:16:20.267510 12119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640: Bootstrap starting.
I20260811 10:16:20.267990 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:20.268396 12119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:20.268741 12119 log.cc:824] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:20.269637 12119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640: No bootstrap required, opened a new log
I20260811 10:16:20.271122 12123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:20.269921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45931 (local address 127.0.0.1:45733)
0811 10:16:20.269981 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:20.269984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:20.270113 (+ 129us) server_negotiation.cc:407] Connection header received
0811 10:16:20.270150 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:20.270152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:20.270157 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:20.270232 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:20.270895 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:20.270897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:20.270898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:20.270902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:20.270920 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:20.270925 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:20.270984 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:20.271036 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 10:16:20.271049 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 10:16:20.272334 12119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 10:16:20.272513 12119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:20.272538 12119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7fefdc2c374a0ba46e9c223fea9640, State: Initialized, Role: FOLLOWER
I20260811 10:16:20.272637 12119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 10:16:20.272760 12119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:20.272797 12119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:20.272854 12119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:20.273548 12119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 10:16:20.273653 12119 leader_election.cc:302] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d7fefdc2c374a0ba46e9c223fea9640; no voters:
I20260811 10:16:20.273885 12119 leader_election.cc:288] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:20.274091 12135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:20.274179 12119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:20.274384 12135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [term 1 LEADER]: Becoming Leader. State: Replica: 4d7fefdc2c374a0ba46e9c223fea9640, State: Running, Role: LEADER
I20260811 10:16:20.274658 12135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } }
I20260811 10:16:20.275342 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d7fefdc2c374a0ba46e9c223fea9640. Latest consensus state: current_term: 1 leader_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260811 10:16:20.275457 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:20.275498 12136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d7fefdc2c374a0ba46e9c223fea9640" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45733 } } }
I20260811 10:16:20.275619 12136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:20.275880 12158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:20.276708 12158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:20.278605 12158 catalog_manager.cc:1347] Generated new cluster ID: 5a346bda2abc4b328e230f8be8884fbd
I20260811 10:16:20.278672 12158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:20.306739 12158 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:20.306990 12158 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:20.313010 12158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d7fefdc2c374a0ba46e9c223fea9640: Generated new TSK 0
I20260811 10:16:20.313452 12158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:20.318447 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 11992
2026-08-11T10:16:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1718 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:20 dist-test-slave-7f01 krb5kdc[12173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:20 dist-test-slave-7f01 krb5kdc[12173](info): set up 2 sockets
Aug 11 10:16:20 dist-test-slave-7f01 krb5kdc[12173](info): commencing operation
krb5kdc: starting...
W20260811 10:16:21.666245 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:21 dist-test-slave-7f01 krb5kdc[12173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:21.828111 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:21.928885 12193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:21.929154 12193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:21.929185 12193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:21.930642 12193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:21.930704 12193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:21.930721 12193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:21.930737 12193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:21.930750 12193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:21.932824 12193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:21.933161 12193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:21.933475 12193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:21.939430 12199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:21.939430 12204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:21.939581 12200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:21.940003 12193 server_base.cc:1034] running on GCE node
I20260811 10:16:21.940387 12193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:21.940811 12193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:21.942020 12193 hybrid_clock.cc:648] HybridClock initialized: now 1786443381941993 us; error 44 us; skew 500 ppm
Aug 11 10:16:21 dist-test-slave-7f01 krb5kdc[12173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:21.945665 12193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:21.946290 12193 webserver.cc:460] Webserver started at http://127.0.0.1:33365/ using document root <none> and password file <none>
I20260811 10:16:21.946714 12193 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:21.946921 12193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:21.947119 12193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:21.948596 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "7a5f0606c2f2456cabc863333bbfbd37"
format_stamp: "Formatted at 2026-08-11 10:16:21 on dist-test-slave-7f01"
server_key: "73515168d803b2fa6153bdbce2c24a18"
server_key_iv: "5cb7c310cc6e07c5cd21f1e27206e41c"
server_key_version: "encryptionkey@0"
I20260811 10:16:21.949051 12193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "7a5f0606c2f2456cabc863333bbfbd37"
format_stamp: "Formatted at 2026-08-11 10:16:21 on dist-test-slave-7f01"
server_key: "73515168d803b2fa6153bdbce2c24a18"
server_key_iv: "5cb7c310cc6e07c5cd21f1e27206e41c"
server_key_version: "encryptionkey@0"
I20260811 10:16:21.950968 12193 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 10:16:21.952379 12213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:21.952649 12193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:21.952805 12193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "7a5f0606c2f2456cabc863333bbfbd37"
format_stamp: "Formatted at 2026-08-11 10:16:21 on dist-test-slave-7f01"
server_key: "73515168d803b2fa6153bdbce2c24a18"
server_key_iv: "5cb7c310cc6e07c5cd21f1e27206e41c"
server_key_version: "encryptionkey@0"
I20260811 10:16:21.952934 12193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:21.976112 12193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:21.979552 12193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:21.979815 12193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:21.984596 12193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44697
I20260811 10:16:21.984640 12311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44697 every 8 connection(s)
I20260811 10:16:21.985057 12193 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:21.986227 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12193
I20260811 10:16:21.986346 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:21.986480 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:21.986591 9144 external_mini_cluster.cc:1428] Setting key 597b7b42f22998d04b799796c8e86032
I20260811 10:16:21.991430 12313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37: Bootstrap starting.
I20260811 10:16:21.992455 12313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:21.992861 12313 log.cc:824] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:21.994087 12313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37: No bootstrap required, opened a new log
Aug 11 10:16:21 dist-test-slave-7f01 krb5kdc[12173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:16:21.997623 12313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } }
I20260811 10:16:21.997884 12313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:21.997931 12313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a5f0606c2f2456cabc863333bbfbd37, State: Initialized, Role: FOLLOWER
I20260811 10:16:21.998097 12313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } }
I20260811 10:16:21.998214 12313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:21.998257 12313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:21.998293 12313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:21.999282 12313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } }
I20260811 10:16:21.999432 12313 leader_election.cc:302] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a5f0606c2f2456cabc863333bbfbd37; no voters:
I20260811 10:16:21.999639 12313 leader_election.cc:288] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:21.999913 12323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:22.000042 12313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:22.000171 12323 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [term 1 LEADER]: Becoming Leader. State: Replica: 7a5f0606c2f2456cabc863333bbfbd37, State: Running, Role: LEADER
I20260811 10:16:22.000319 12323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } }
I20260811 10:16:22.000946 12326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a5f0606c2f2456cabc863333bbfbd37. Latest consensus state: current_term: 1 leader_uuid: "7a5f0606c2f2456cabc863333bbfbd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } } }
I20260811 10:16:22.001072 12326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:22.001395 12342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:22.001528 12324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a5f0606c2f2456cabc863333bbfbd37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a5f0606c2f2456cabc863333bbfbd37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44697 } } }
I20260811 10:16:22.002045 12324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:22.002537 12342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:22.003530 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:21.987475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54951 (local address 127.0.0.1:44697)
0811 10:16:21.987788 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:21.987804 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:21.987859 (+ 55us) server_negotiation.cc:407] Connection header received
0811 10:16:21.988863 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:21.988877 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:21.988967 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:21.989203 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:21.989942 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:21.991591 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:21.993186 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:21.993890 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:21.994061 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:21.997859 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:21.997866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:21.997866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:21.997885 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:22.000365 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:22.001152 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:22.001157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:22.001158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:22.001347 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:22.001606 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:22.001607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:22.001607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:22.002025 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:22.002361 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:22.003091 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 10:16:22.003278 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":203,"thread_start_us":118,"threads_started":1}
I20260811 10:16:22.005057 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:22.005288 12342 catalog_manager.cc:1347] Generated new cluster ID: 1a34934de3b34a659d34689b75b38411
I20260811 10:16:22.005352 12342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:22.018133 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:22.008305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34181 (local address 127.0.0.1:44697)
0811 10:16:22.008361 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:22.008364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:22.010206 (+ 1842us) server_negotiation.cc:407] Connection header received
0811 10:16:22.010304 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:22.010305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:22.010318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:22.010395 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:22.011086 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:22.011659 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:22.012510 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:22.012773 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:22.012865 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:22.014341 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:22.014343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:22.014344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:22.014353 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:22.016128 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:22.016546 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:22.016548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:22.016548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:22.016617 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:22.017223 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:22.017225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:22.017225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:22.017420 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:22.017562 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:22.017759 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 10:16:22.017933 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:16:22.032097 12342 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:22.032848 12342 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:22.039245 12342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a5f0606c2f2456cabc863333bbfbd37: Generated new TSK 0
I20260811 10:16:22.039533 12342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:22.050941 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12193
2026-08-11T10:16:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1729 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:22 dist-test-slave-7f01 krb5kdc[12375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:22 dist-test-slave-7f01 krb5kdc[12375](info): set up 2 sockets
Aug 11 10:16:22 dist-test-slave-7f01 krb5kdc[12375](info): commencing operation
krb5kdc: starting...
W20260811 10:16:23.393252 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:23 dist-test-slave-7f01 krb5kdc[12375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:23.548503 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:23.646535 12397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:23.646914 12397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:23.646963 12397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:23.649266 12397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:23.649427 12397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:23.649458 12397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:23.649484 12397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:23.649516 12397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:23.651556 12397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:23.651957 12397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:23.652244 12397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:23.657934 12407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:23.657977 12408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:23.657977 12412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:23.658437 12397 server_base.cc:1034] running on GCE node
I20260811 10:16:23.658766 12397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:23.659150 12397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:23.660351 12397 hybrid_clock.cc:648] HybridClock initialized: now 1786443383660327 us; error 38 us; skew 500 ppm
Aug 11 10:16:23 dist-test-slave-7f01 krb5kdc[12375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:23.664629 12397 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:23.665387 12397 webserver.cc:460] Webserver started at http://127.0.0.1:41719/ using document root <none> and password file <none>
I20260811 10:16:23.665699 12397 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:23.665777 12397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:23.665952 12397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:23.667302 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "a39275aadaaf4f88adfd69d927bf184c"
format_stamp: "Formatted at 2026-08-11 10:16:23 on dist-test-slave-7f01"
server_key: "349f8f6ae4bda212ae1131ad2e655782"
server_key_iv: "f4833c82bb9039142d294243744d164d"
server_key_version: "encryptionkey@0"
I20260811 10:16:23.667810 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "a39275aadaaf4f88adfd69d927bf184c"
format_stamp: "Formatted at 2026-08-11 10:16:23 on dist-test-slave-7f01"
server_key: "349f8f6ae4bda212ae1131ad2e655782"
server_key_iv: "f4833c82bb9039142d294243744d164d"
server_key_version: "encryptionkey@0"
I20260811 10:16:23.670012 12397 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:16:23.671052 12429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:23.671420 12397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:23.671612 12397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "a39275aadaaf4f88adfd69d927bf184c"
format_stamp: "Formatted at 2026-08-11 10:16:23 on dist-test-slave-7f01"
server_key: "349f8f6ae4bda212ae1131ad2e655782"
server_key_iv: "f4833c82bb9039142d294243744d164d"
server_key_version: "encryptionkey@0"
I20260811 10:16:23.671716 12397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:23.687192 12397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:23.689858 12397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:23.690060 12397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:23.695210 12397 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35891
I20260811 10:16:23.695377 12519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35891 every 8 connection(s)
I20260811 10:16:23.695788 12397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:23.696346 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12397
I20260811 10:16:23.696476 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:23.696691 9144 external_mini_cluster.cc:1428] Setting key 1eb5a540ce978838843b1b87044f7da8
I20260811 10:16:23.698199 12520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:23.702236 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c: Bootstrap starting.
I20260811 10:16:23.703217 12520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:23.703603 12520 log.cc:824] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:23.704172 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:23.697439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50925 (local address 127.0.0.1:35891)
0811 10:16:23.697715 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:23.697726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:23.697759 (+ 33us) server_negotiation.cc:407] Connection header received
0811 10:16:23.698490 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:23.698500 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:23.698563 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:23.698715 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:23.699094 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:23.700175 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:23.701550 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:23.702339 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:23.702552 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:23.703403 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:23.703408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:23.703408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:23.703594 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:23.703635 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:23.703648 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:23.703727 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:23.703840 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 10:16:23.703922 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":155,"threads_started":1}
I20260811 10:16:23.704583 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c: No bootstrap required, opened a new log
I20260811 10:16:23.705302 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:23.706894 12520 raft_consensus.cc:348] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } }
I20260811 10:16:23.707160 12520 raft_consensus.cc:374] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:23.707242 12520 raft_consensus.cc:729] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a39275aadaaf4f88adfd69d927bf184c, State: Initialized, Role: FOLLOWER
I20260811 10:16:23.707389 12520 consensus_queue.cc:260] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } }
I20260811 10:16:23.707476 12520 raft_consensus.cc:388] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:23.707527 12520 raft_consensus.cc:482] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:23.707581 12520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:23.708364 12520 raft_consensus.cc:504] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } }
I20260811 10:16:23.708498 12520 leader_election.cc:302] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a39275aadaaf4f88adfd69d927bf184c; no voters:
I20260811 10:16:23.708720 12520 leader_election.cc:288] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:23.708779 12538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:23.708953 12538 raft_consensus.cc:686] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [term 1 LEADER]: Becoming Leader. State: Replica: a39275aadaaf4f88adfd69d927bf184c, State: Running, Role: LEADER
I20260811 10:16:23.709080 12520 sys_catalog.cc:564] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:23.709064 12538 consensus_queue.cc:237] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } }
I20260811 10:16:23.709491 12544 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a39275aadaaf4f88adfd69d927bf184c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } } }
I20260811 10:16:23.709591 12544 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:23.709767 12544 sys_catalog.cc:455] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a39275aadaaf4f88adfd69d927bf184c. Latest consensus state: current_term: 1 leader_uuid: "a39275aadaaf4f88adfd69d927bf184c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a39275aadaaf4f88adfd69d927bf184c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35891 } } }
I20260811 10:16:23.709849 12544 sys_catalog.cc:458] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:23.710000 12549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:23.710695 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:23.706976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54567 (local address 127.0.0.1:35891)
0811 10:16:23.707018 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:23.707020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:23.707075 (+ 55us) server_negotiation.cc:407] Connection header received
0811 10:16:23.707148 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:23.707164 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:23.707177 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:23.707237 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:23.707413 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:23.708098 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:23.709106 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:23.709416 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:23.709466 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:23.710388 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:23.710389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:23.710390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:23.710396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:23.710413 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:23.710417 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:23.710446 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:23.710558 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 10:16:23.710589 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:16:23.710618 12549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:23.713641 12549 catalog_manager.cc:1347] Generated new cluster ID: a6a1020ef8884414bd6ac962a9f6adb0
I20260811 10:16:23.713727 12549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:23.735821 12549 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:23.736701 12549 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:23.740664 12549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a39275aadaaf4f88adfd69d927bf184c: Generated new TSK 0
I20260811 10:16:23.740912 12549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:23.746577 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12397
2026-08-11T10:16:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1692 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:23 dist-test-slave-7f01 krb5kdc[12574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:23 dist-test-slave-7f01 krb5kdc[12574](info): set up 2 sockets
Aug 11 10:16:23 dist-test-slave-7f01 krb5kdc[12574](info): commencing operation
krb5kdc: starting...
W20260811 10:16:25.111867 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:25 dist-test-slave-7f01 krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:25.268333 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:25.364584 12602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:25.364892 12602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:25.364938 12602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:25.366349 12602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:25.366451 12602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:25.366477 12602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:25.366501 12602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:25.366523 12602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:25.368464 12602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:25.368842 12602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:25.369119 12602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:25.375106 12614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:25.375106 12612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:25.375381 12602 server_base.cc:1034] running on GCE node
W20260811 10:16:25.375141 12611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:25.375975 12602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:25.376363 12602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:25.377553 12602 hybrid_clock.cc:648] HybridClock initialized: now 1786443385377511 us; error 67 us; skew 500 ppm
Aug 11 10:16:25 dist-test-slave-7f01 krb5kdc[12574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:25.384934 12602 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:25.385558 12602 webserver.cc:460] Webserver started at http://127.0.0.1:44247/ using document root <none> and password file <none>
I20260811 10:16:25.386009 12602 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:25.386176 12602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:25.386382 12602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:25.387888 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "1e50ca7b234f4d02bb341bab64bdece3"
format_stamp: "Formatted at 2026-08-11 10:16:25 on dist-test-slave-7f01"
server_key: "b199ce99abf32eb09727d2fff3d9098a"
server_key_iv: "523565fd6f0f5bfefb963ed569495850"
server_key_version: "encryptionkey@0"
I20260811 10:16:25.388355 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "1e50ca7b234f4d02bb341bab64bdece3"
format_stamp: "Formatted at 2026-08-11 10:16:25 on dist-test-slave-7f01"
server_key: "b199ce99abf32eb09727d2fff3d9098a"
server_key_iv: "523565fd6f0f5bfefb963ed569495850"
server_key_version: "encryptionkey@0"
I20260811 10:16:25.390283 12602 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:25.391530 12626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:25.391983 12602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:25.392168 12602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "1e50ca7b234f4d02bb341bab64bdece3"
format_stamp: "Formatted at 2026-08-11 10:16:25 on dist-test-slave-7f01"
server_key: "b199ce99abf32eb09727d2fff3d9098a"
server_key_iv: "523565fd6f0f5bfefb963ed569495850"
server_key_version: "encryptionkey@0"
I20260811 10:16:25.392280 12602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:25.396867 12602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:25.399708 12602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:25.399987 12602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:25.405490 12602 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39887
I20260811 10:16:25.405577 12716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39887 every 8 connection(s)
I20260811 10:16:25.405974 12602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:25.407164 12717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:25.409924 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: Bootstrap starting.
I20260811 10:16:25.410599 12717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:25.410866 12717 log.cc:824] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:25.411693 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: No bootstrap required, opened a new log
I20260811 10:16:25.413540 12717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260811 10:16:25.413725 12717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:25.413749 12717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e50ca7b234f4d02bb341bab64bdece3, State: Initialized, Role: FOLLOWER
I20260811 10:16:25.413918 12717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260811 10:16:25.414014 12717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:25.414040 12717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:25.414072 12717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:25.414691 12717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260811 10:16:25.414791 12717 leader_election.cc:302] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e50ca7b234f4d02bb341bab64bdece3; no voters:
I20260811 10:16:25.414961 12717 leader_election.cc:288] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:25.415177 12722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:25.415295 12717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:25.415504 12722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [term 1 LEADER]: Becoming Leader. State: Replica: 1e50ca7b234f4d02bb341bab64bdece3, State: Running, Role: LEADER
I20260811 10:16:25.415669 12722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260811 10:16:25.416247 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12602
I20260811 10:16:25.416401 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:25.416682 9144 external_mini_cluster.cc:1428] Setting key 9bb3e4b381d9049abd0df8d5d9f323a0
I20260811 10:16:25.416996 12729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e50ca7b234f4d02bb341bab64bdece3. Latest consensus state: current_term: 1 leader_uuid: "1e50ca7b234f4d02bb341bab64bdece3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } } }
I20260811 10:16:25.417145 12729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:25.417438 12723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e50ca7b234f4d02bb341bab64bdece3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e50ca7b234f4d02bb341bab64bdece3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } } }
I20260811 10:16:25.417778 12723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3 [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:25.418326 12747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:25.418416 12747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:25.418538 12748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:25.418787 12748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:25.420449 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:25.417531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40737 (local address 127.0.0.1:39887)
0811 10:16:25.417875 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:25.417887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:25.418512 (+ 625us) server_negotiation.cc:407] Connection header received
0811 10:16:25.418663 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:25.418672 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:25.418711 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:25.418916 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:25.419877 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:25.419881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:25.419882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:25.419887 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:25.419947 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:25.419959 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:25.420036 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:25.420211 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 10:16:25.420257 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":86,"threads_started":1}
I20260811 10:16:25.421530 12748 catalog_manager.cc:1347] Generated new cluster ID: 97db5d254f994d0f925b5c1413bee208
I20260811 10:16:25.421602 12748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:25.421681 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:25.424496 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:25.423194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55239 (local address 127.0.0.1:39887)
0811 10:16:25.423238 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:25.423241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:25.423372 (+ 131us) server_negotiation.cc:407] Connection header received
0811 10:16:25.423457 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:25.423458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:25.423467 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:25.423524 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:25.424222 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:25.424224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:25.424224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:25.424229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:25.424245 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:25.424250 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:25.424309 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:25.424366 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 10:16:25.424382 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:16:25.429044 12748 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:25.429263 12748 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:25.436347 12748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e50ca7b234f4d02bb341bab64bdece3: Generated new TSK 0
I20260811 10:16:25.436655 12748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:25.446012 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12602
2026-08-11T10:16:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1691 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:25 dist-test-slave-7f01 krb5kdc[12769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:25 dist-test-slave-7f01 krb5kdc[12769](info): set up 2 sockets
Aug 11 10:16:25 dist-test-slave-7f01 krb5kdc[12769](info): commencing operation
krb5kdc: starting...
W20260811 10:16:26.813736 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:26 dist-test-slave-7f01 krb5kdc[12769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:26.976038 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:27.077948 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:27.078230 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:27.078258 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:27.079500 12790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:27.079550 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:27.079566 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:27.079579 12790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:27.079591 12790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:27.081379 12790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:27.081702 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:27.081997 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:27.087720 12802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:27.087759 12798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:27.087760 12797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:27.087987 12790 server_base.cc:1034] running on GCE node
I20260811 10:16:27.088814 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:27.089262 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:27.090502 12790 hybrid_clock.cc:648] HybridClock initialized: now 1786443387090409 us; error 130 us; skew 500 ppm
Aug 11 10:16:27 dist-test-slave-7f01 krb5kdc[12769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:27.094345 12790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:27.095167 12790 webserver.cc:460] Webserver started at http://127.0.0.1:35439/ using document root <none> and password file <none>
I20260811 10:16:27.095484 12790 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:27.095590 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:27.095851 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:27.097304 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5"
format_stamp: "Formatted at 2026-08-11 10:16:27 on dist-test-slave-7f01"
server_key: "b1f2b94804ad95f5e112d6986993468e"
server_key_iv: "802c0a09b9ce37e3e0f4b35cc9deb012"
server_key_version: "encryptionkey@0"
I20260811 10:16:27.097805 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5"
format_stamp: "Formatted at 2026-08-11 10:16:27 on dist-test-slave-7f01"
server_key: "b1f2b94804ad95f5e112d6986993468e"
server_key_iv: "802c0a09b9ce37e3e0f4b35cc9deb012"
server_key_version: "encryptionkey@0"
I20260811 10:16:27.099907 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 10:16:27.101042 12810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:27.101476 12790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:27.101641 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5"
format_stamp: "Formatted at 2026-08-11 10:16:27 on dist-test-slave-7f01"
server_key: "b1f2b94804ad95f5e112d6986993468e"
server_key_iv: "802c0a09b9ce37e3e0f4b35cc9deb012"
server_key_version: "encryptionkey@0"
I20260811 10:16:27.101750 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:27.136155 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:27.139086 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:27.139338 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:27.145504 12790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45039
I20260811 10:16:27.145668 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45039 every 8 connection(s)
I20260811 10:16:27.146023 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:27.147173 12904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:27.149943 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5: Bootstrap starting.
I20260811 10:16:27.150636 12904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:27.150980 12904 log.cc:824] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:27.151418 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12790
I20260811 10:16:27.151518 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:27.151810 9144 external_mini_cluster.cc:1428] Setting key 9bd893622e87bfdfcb38fcb243b96ca4
I20260811 10:16:27.152128 12904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5: No bootstrap required, opened a new log
I20260811 10:16:27.154629 12904 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } }
I20260811 10:16:27.154853 12904 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:27.154911 12904 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6764ae5f7e466a84f2e4ac10e7f6f5, State: Initialized, Role: FOLLOWER
I20260811 10:16:27.155115 12904 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } }
I20260811 10:16:27.155236 12904 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:27.155288 12904 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:27.155346 12904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:27.156186 12904 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } }
I20260811 10:16:27.156339 12904 leader_election.cc:302] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea6764ae5f7e466a84f2e4ac10e7f6f5; no voters:
I20260811 10:16:27.156569 12904 leader_election.cc:288] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:27.156636 12911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:27.156864 12911 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [term 1 LEADER]: Becoming Leader. State: Replica: ea6764ae5f7e466a84f2e4ac10e7f6f5, State: Running, Role: LEADER
I20260811 10:16:27.156975 12904 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:27.157037 12911 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } }
I20260811 10:16:27.157532 12911 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea6764ae5f7e466a84f2e4ac10e7f6f5. Latest consensus state: current_term: 1 leader_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } } }
I20260811 10:16:27.157542 12915 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6764ae5f7e466a84f2e4ac10e7f6f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45039 } } }
I20260811 10:16:27.157711 12911 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:27.157719 12915 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:27.158026 12927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:27.158824 12927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:27.160950 12927 catalog_manager.cc:1347] Generated new cluster ID: 51f6632922954191a4e91c2ecdedf1fd
I20260811 10:16:27.161016 12927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:16:27 dist-test-slave-7f01 krb5kdc[12769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:16:27.165793 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:27.152472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:44231 (local address 127.0.0.1:45039)
0811 10:16:27.152715 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:27.152727 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:27.152762 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:16:27.153754 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:27.153764 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:27.153844 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:27.154035 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:27.154844 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:27.156512 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:27.157825 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:27.158635 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:27.158766 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:27.162064 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:27.162069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:27.162070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:27.162091 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:27.164061 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:27.164547 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:27.164551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:27.164551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:27.164648 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:27.164903 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:27.164904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:27.164904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:27.165183 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:27.165365 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:27.165484 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 10:16:27.165598 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":113,"threads_started":1}
I20260811 10:16:27.167225 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:27.178663 12927 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:27.179513 12927 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:27.180044 12909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:27.169645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:53187 (local address 127.0.0.1:45039)
0811 10:16:27.169730 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:27.169733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:27.172075 (+ 2342us) server_negotiation.cc:407] Connection header received
0811 10:16:27.172168 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:27.172170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:27.172185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:27.172251 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:27.172974 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:27.173665 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:27.174489 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:27.174764 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:27.174855 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:27.176447 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:27.176448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:27.176449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:27.176458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:27.178151 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:27.178939 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:27.178940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:27.178940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:27.179011 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:27.179322 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:27.179323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:27.179323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:27.179468 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:27.179602 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:27.179863 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 10:16:27.179940 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 10:16:27.195777 12927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea6764ae5f7e466a84f2e4ac10e7f6f5: Generated new TSK 0
I20260811 10:16:27.196036 12927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:27.206193 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12790
W20260811 10:16:27.241706 12761 connection.cc:535] client connection to 127.0.0.1:45039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45039): Connection reset by peer (error 104)
2026-08-11T10:16:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1781 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:27 dist-test-slave-7f01 krb5kdc[12961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:27 dist-test-slave-7f01 krb5kdc[12961](info): set up 2 sockets
Aug 11 10:16:27 dist-test-slave-7f01 krb5kdc[12961](info): commencing operation
krb5kdc: starting...
W20260811 10:16:28.584106 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:28 dist-test-slave-7f01 krb5kdc[12961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:28.745569 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:28.842164 12982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:28.842443 12982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:28.842489 12982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:28.843784 12982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:28.843864 12982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:28.843889 12982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:28.843911 12982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:28.843940 12982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:28.845674 12982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:28.846027 12982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:28.846297 12982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:28.852128 12992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:28.852212 12990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:28.852102 12994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:28.852698 12982 server_base.cc:1034] running on GCE node
I20260811 10:16:28.853034 12982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:28.853425 12982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:28.854660 12982 hybrid_clock.cc:648] HybridClock initialized: now 1786443388854628 us; error 39 us; skew 500 ppm
Aug 11 10:16:28 dist-test-slave-7f01 krb5kdc[12961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443388, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:28.858348 12982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:28.859004 12982 webserver.cc:460] Webserver started at http://127.0.0.1:40489/ using document root <none> and password file <none>
I20260811 10:16:28.859344 12982 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:28.859400 12982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:28.859567 12982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:28.860965 12982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "ae4e14e792d24e49a94493d26b850ab3"
format_stamp: "Formatted at 2026-08-11 10:16:28 on dist-test-slave-7f01"
server_key: "9f0edac34d22ef07385690ee32c51740"
server_key_iv: "a40f3b6227660ee69a47f28d6a4e5ae1"
server_key_version: "encryptionkey@0"
I20260811 10:16:28.861418 12982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "ae4e14e792d24e49a94493d26b850ab3"
format_stamp: "Formatted at 2026-08-11 10:16:28 on dist-test-slave-7f01"
server_key: "9f0edac34d22ef07385690ee32c51740"
server_key_iv: "a40f3b6227660ee69a47f28d6a4e5ae1"
server_key_version: "encryptionkey@0"
I20260811 10:16:28.863554 12982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 10:16:28.864804 13008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:28.865056 12982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:28.865187 12982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "ae4e14e792d24e49a94493d26b850ab3"
format_stamp: "Formatted at 2026-08-11 10:16:28 on dist-test-slave-7f01"
server_key: "9f0edac34d22ef07385690ee32c51740"
server_key_iv: "a40f3b6227660ee69a47f28d6a4e5ae1"
server_key_version: "encryptionkey@0"
I20260811 10:16:28.865278 12982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:28.878201 12982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:28.881232 12982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:28.881403 12982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:28.887003 12982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46163
I20260811 10:16:28.887108 13100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46163 every 8 connection(s)
I20260811 10:16:28.887578 12982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:28.888811 13102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:28.893028 13102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3: Bootstrap starting.
I20260811 10:16:28.893953 13102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:28.894338 13102 log.cc:824] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:28.894721 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 12982
I20260811 10:16:28.894887 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:28.895187 9144 external_mini_cluster.cc:1428] Setting key b524f0e96708c52d127cbac418ef3d6a
I20260811 10:16:28.895807 13102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3: No bootstrap required, opened a new log
I20260811 10:16:28.897989 13102 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } }
I20260811 10:16:28.898180 13102 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:28.898204 13102 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4e14e792d24e49a94493d26b850ab3, State: Initialized, Role: FOLLOWER
I20260811 10:16:28.898375 13102 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } }
I20260811 10:16:28.898489 13102 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:28.898561 13102 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:28.898624 13102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:28.899462 13102 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } }
I20260811 10:16:28.899617 13102 leader_election.cc:302] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4e14e792d24e49a94493d26b850ab3; no voters:
I20260811 10:16:28.899901 13102 leader_election.cc:288] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:28.899945 13114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:28.900101 13114 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [term 1 LEADER]: Becoming Leader. State: Replica: ae4e14e792d24e49a94493d26b850ab3, State: Running, Role: LEADER
I20260811 10:16:28.900259 13102 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:28.900285 13114 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } }
I20260811 10:16:28.900794 13115 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4e14e792d24e49a94493d26b850ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } } }
I20260811 10:16:28.900966 13115 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:28.901747 13130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:28.902115 13118 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4e14e792d24e49a94493d26b850ab3. Latest consensus state: current_term: 1 leader_uuid: "ae4e14e792d24e49a94493d26b850ab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4e14e792d24e49a94493d26b850ab3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46163 } } }
I20260811 10:16:28.902308 13118 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:28.902413 13130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:28.904958 13130 catalog_manager.cc:1347] Generated new cluster ID: 33d3c5e234514c42bb00f63ba6d2bae8
I20260811 10:16:28.905025 13130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:28.905443 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:28.896104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:41761 (local address 127.0.0.1:46163)
0811 10:16:28.896354 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:28.896364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:28.896401 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:16:28.897959 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:28.897970 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:28.898217 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:28.898490 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:28.898843 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:28.900561 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:28.902247 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:28.903177 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:28.903353 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:28.904479 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:28.904484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:28.904485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:28.904758 (+ 273us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:28.904812 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:28.904844 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:28.905018 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:28.905169 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 10:16:28.905204 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":119,"threads_started":1}
I20260811 10:16:28.906878 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:28.916841 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:28.912367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:59575 (local address 127.0.0.1:46163)
0811 10:16:28.912452 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:28.912455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:28.912682 (+ 227us) server_negotiation.cc:407] Connection header received
0811 10:16:28.912766 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:28.912767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:28.912782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:28.912866 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:28.913106 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:28.913899 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:28.915121 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:28.915476 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:28.915545 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:28.916521 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:28.916523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:28.916523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:28.916530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:28.916549 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:28.916554 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:28.916609 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:28.916720 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 10:16:28.916739 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 10:16:28.919323 13130 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:28.920027 13130 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:28.923688 13130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae4e14e792d24e49a94493d26b850ab3: Generated new TSK 0
I20260811 10:16:28.923990 13130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:28.928905 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 12982
W20260811 10:16:28.952471 12959 connection.cc:535] client connection to 127.0.0.1:46163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46163): Connection reset by peer (error 104)
2026-08-11T10:16:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:29 dist-test-slave-7f01 krb5kdc[13155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:29 dist-test-slave-7f01 krb5kdc[13155](info): set up 2 sockets
Aug 11 10:16:29 dist-test-slave-7f01 krb5kdc[13155](info): commencing operation
krb5kdc: starting...
W20260811 10:16:30.306849 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:30 dist-test-slave-7f01 krb5kdc[13155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:30.464239 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:30.558506 13181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:30.558817 13181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:30.558844 13181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:30.560530 13181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:30.560619 13181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:30.560636 13181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:30.560650 13181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:30.560664 13181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:30.562616 13181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:30.563011 13181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:30.563436 13181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:30.569900 13190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:30.569974 13194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:30.569900 13191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:30.570317 13181 server_base.cc:1034] running on GCE node
I20260811 10:16:30.570642 13181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:30.571023 13181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:30.572281 13181 hybrid_clock.cc:648] HybridClock initialized: now 1786443390572249 us; error 35 us; skew 500 ppm
Aug 11 10:16:30 dist-test-slave-7f01 krb5kdc[13155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:30.579399 13181 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:30.580204 13181 webserver.cc:460] Webserver started at http://127.0.0.1:35429/ using document root <none> and password file <none>
I20260811 10:16:30.580521 13181 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:30.580602 13181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:30.580794 13181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:30.582150 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "907720a36dac4a4e947e17bbdf8fe4c5"
format_stamp: "Formatted at 2026-08-11 10:16:30 on dist-test-slave-7f01"
server_key: "57e3bc79623f46a63aae2fa8a9a515dc"
server_key_iv: "57aa743369e305ae99222c14984d5881"
server_key_version: "encryptionkey@0"
I20260811 10:16:30.582607 13181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "907720a36dac4a4e947e17bbdf8fe4c5"
format_stamp: "Formatted at 2026-08-11 10:16:30 on dist-test-slave-7f01"
server_key: "57e3bc79623f46a63aae2fa8a9a515dc"
server_key_iv: "57aa743369e305ae99222c14984d5881"
server_key_version: "encryptionkey@0"
I20260811 10:16:30.584640 13181 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260811 10:16:30.585950 13205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:30.586246 13181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:30.586395 13181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "907720a36dac4a4e947e17bbdf8fe4c5"
format_stamp: "Formatted at 2026-08-11 10:16:30 on dist-test-slave-7f01"
server_key: "57e3bc79623f46a63aae2fa8a9a515dc"
server_key_iv: "57aa743369e305ae99222c14984d5881"
server_key_version: "encryptionkey@0"
I20260811 10:16:30.586508 13181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:30.590550 13181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:30.593362 13181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:30.593554 13181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:30.598616 13181 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34091
I20260811 10:16:30.598657 13286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34091 every 8 connection(s)
I20260811 10:16:30.599047 13181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:30.600275 13287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:30.602205 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13181
I20260811 10:16:30.602321 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:30.602550 9144 external_mini_cluster.cc:1428] Setting key 7dc9965348156c8c10840582838f3ff6
I20260811 10:16:30.603339 13287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5: Bootstrap starting.
I20260811 10:16:30.604133 13287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:30.604463 13287 log.cc:824] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:30.605475 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:30.603224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:58317 (local address 127.0.0.1:34091)
0811 10:16:30.603464 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:30.603474 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:30.603503 (+ 29us) server_negotiation.cc:407] Connection header received
0811 10:16:30.604053 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:30.604062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:30.604090 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:30.604262 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:30.604984 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:30.604988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:30.604988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:30.604995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:30.605045 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:30.605056 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:30.605155 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:30.605213 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 10:16:30.605265 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":115,"threads_started":1}
I20260811 10:16:30.605568 13287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5: No bootstrap required, opened a new log
I20260811 10:16:30.606769 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:30.608430 13287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } }
I20260811 10:16:30.608672 13287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:30.608743 13287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907720a36dac4a4e947e17bbdf8fe4c5, State: Initialized, Role: FOLLOWER
I20260811 10:16:30.608897 13287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } }
I20260811 10:16:30.609005 13287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:30.609052 13287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:30.609107 13287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:30.609753 13287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } }
I20260811 10:16:30.609896 13287 leader_election.cc:302] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 907720a36dac4a4e947e17bbdf8fe4c5; no voters:
I20260811 10:16:30.610098 13287 leader_election.cc:288] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:30.610206 13304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:30.610383 13304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 907720a36dac4a4e947e17bbdf8fe4c5, State: Running, Role: LEADER
I20260811 10:16:30.610387 13287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:30.610704 13304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } }
I20260811 10:16:30.611657 13309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 907720a36dac4a4e947e17bbdf8fe4c5. Latest consensus state: current_term: 1 leader_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } } }
I20260811 10:16:30.611779 13293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:30.608715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:43381 (local address 127.0.0.1:34091)
0811 10:16:30.608764 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:30.608766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:30.610545 (+ 1779us) server_negotiation.cc:407] Connection header received
0811 10:16:30.610631 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:30.610632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:30.610637 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:30.610725 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:30.611438 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:30.611439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:30.611440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:30.611445 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:30.611465 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:30.611470 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:30.611524 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:30.611650 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 10:16:30.611664 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:16:30.611862 13309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:30.612143 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "907720a36dac4a4e947e17bbdf8fe4c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34091 } } }
I20260811 10:16:30.612303 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:30.612398 13318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:30.612970 13318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:30.615253 13318 catalog_manager.cc:1347] Generated new cluster ID: 7247e0aa25054b5e8e42a439573d57e4
I20260811 10:16:30.615334 13318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:30.629488 13318 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:30.629832 13318 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:30.635191 13318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 907720a36dac4a4e947e17bbdf8fe4c5: Generated new TSK 0
I20260811 10:16:30.635623 13318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:30.641301 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13181
2026-08-11T10:16:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:30 dist-test-slave-7f01 krb5kdc[13340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:30 dist-test-slave-7f01 krb5kdc[13340](info): set up 2 sockets
Aug 11 10:16:30 dist-test-slave-7f01 krb5kdc[13340](info): commencing operation
krb5kdc: starting...
W20260811 10:16:32.070221 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.393s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:32.228253 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:32.320866 13370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:32.321141 13370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:32.321168 13370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:32.322360 13370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:32.322403 13370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:32.322418 13370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:32.322430 13370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:32.322443 13370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:32.324007 13370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:32.324249 13370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:32.324450 13370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:32.330039 13384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:32.330153 13377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:32.330098 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:32.330580 13370 server_base.cc:1034] running on GCE node
I20260811 10:16:32.330930 13370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:32.331233 13370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:32.332396 13370 hybrid_clock.cc:648] HybridClock initialized: now 1786443392332374 us; error 34 us; skew 500 ppm
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:32.336045 13370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:32.336988 13370 webserver.cc:460] Webserver started at http://127.0.0.1:34837/ using document root <none> and password file <none>
I20260811 10:16:32.337338 13370 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:32.337428 13370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:32.337620 13370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:32.339054 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "e41a43e9cad64b4ca2e94cc1229424da"
format_stamp: "Formatted at 2026-08-11 10:16:32 on dist-test-slave-7f01"
server_key: "ff2359bf00f33cc3a9b31f84bf7e92a9"
server_key_iv: "becbdd9284b043d3efce5a995a1793dd"
server_key_version: "encryptionkey@0"
I20260811 10:16:32.339545 13370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "e41a43e9cad64b4ca2e94cc1229424da"
format_stamp: "Formatted at 2026-08-11 10:16:32 on dist-test-slave-7f01"
server_key: "ff2359bf00f33cc3a9b31f84bf7e92a9"
server_key_iv: "becbdd9284b043d3efce5a995a1793dd"
server_key_version: "encryptionkey@0"
I20260811 10:16:32.341750 13370 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:16:32.343163 13399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:32.343550 13370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:32.343808 13370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "e41a43e9cad64b4ca2e94cc1229424da"
format_stamp: "Formatted at 2026-08-11 10:16:32 on dist-test-slave-7f01"
server_key: "ff2359bf00f33cc3a9b31f84bf7e92a9"
server_key_iv: "becbdd9284b043d3efce5a995a1793dd"
server_key_version: "encryptionkey@0"
I20260811 10:16:32.343951 13370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:32.356117 13370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:32.359177 13370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:32.359383 13370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:32.364732 13370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41817
I20260811 10:16:32.364777 13490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41817 every 8 connection(s)
I20260811 10:16:32.365231 13370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:32.365836 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13370
I20260811 10:16:32.365986 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:32.366246 9144 external_mini_cluster.cc:1428] Setting key d50973952ad916e9839935ae9554b883
I20260811 10:16:32.366675 13491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:32.370322 13491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da: Bootstrap starting.
I20260811 10:16:32.371206 13491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:32.371595 13491 log.cc:824] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:32.373245 13491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da: No bootstrap required, opened a new log
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 10:16:32.376014 13491 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260811 10:16:32.376199 13491 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:32.376224 13491 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41a43e9cad64b4ca2e94cc1229424da, State: Initialized, Role: FOLLOWER
I20260811 10:16:32.376386 13491 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260811 10:16:32.376495 13491 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:32.376535 13491 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:32.376597 13491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:32.377547 13491 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260811 10:16:32.377691 13491 leader_election.cc:302] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41a43e9cad64b4ca2e94cc1229424da; no voters:
I20260811 10:16:32.377920 13491 leader_election.cc:288] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:32.378072 13499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:32.378329 13491 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:32.378501 13499 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [term 1 LEADER]: Becoming Leader. State: Replica: e41a43e9cad64b4ca2e94cc1229424da, State: Running, Role: LEADER
I20260811 10:16:32.378720 13499 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } }
I20260811 10:16:32.379388 13502 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41a43e9cad64b4ca2e94cc1229424da. Latest consensus state: current_term: 1 leader_uuid: "e41a43e9cad64b4ca2e94cc1229424da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } } }
I20260811 10:16:32.379506 13502 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:32.379575 13500 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41a43e9cad64b4ca2e94cc1229424da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41a43e9cad64b4ca2e94cc1229424da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41817 } } }
I20260811 10:16:32.379709 13500 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:32.380004 13517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:32.380963 13517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:32.382162 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:32.366900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:55653 (local address 127.0.0.1:41817)
0811 10:16:32.367126 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:32.367134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:32.367162 (+ 28us) server_negotiation.cc:407] Connection header received
0811 10:16:32.368283 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:32.368293 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:32.368356 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:32.368488 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:32.369442 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:32.370409 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:32.371569 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:32.372162 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:32.372299 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:32.376388 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:32.376393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:32.376394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:32.376410 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:32.379240 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:32.380041 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:32.380046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:32.380047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:32.380185 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:32.380510 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:32.380513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:32.380513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:32.380957 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:32.381203 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:32.381801 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 10:16:32.381929 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":111,"threads_started":1}
I20260811 10:16:32.383718 13517 catalog_manager.cc:1347] Generated new cluster ID: f081d62968bc4c38837a0f399f5be799
I20260811 10:16:32.383704 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:32.383805 13517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:32.393586 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:32.385519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:33607 (local address 127.0.0.1:41817)
0811 10:16:32.385569 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:32.385571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:32.385713 (+ 142us) server_negotiation.cc:407] Connection header received
0811 10:16:32.385764 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:32.385765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:32.385777 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:32.385851 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:32.386537 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:32.387350 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:32.388432 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:32.388784 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:32.388858 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:32.390409 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:32.390411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:32.390412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:32.390420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:32.392220 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:32.392678 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:32.392680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:32.392680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:32.392750 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:32.392999 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:32.393000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:32.393001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:32.393158 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:32.393279 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:32.393405 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 10:16:32.393479 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:16:32.411629 13517 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:32.412467 13517 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:32.432008 13517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41a43e9cad64b4ca2e94cc1229424da: Generated new TSK 0
I20260811 10:16:32.432265 13517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:32.441641 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13370
W20260811 10:16:32.469898 13337 connection.cc:535] client connection to 127.0.0.1:41817 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41817): Connection reset by peer (error 104)
2026-08-11T10:16:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1807 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13537](info): set up 2 sockets
Aug 11 10:16:32 dist-test-slave-7f01 krb5kdc[13537](info): commencing operation
krb5kdc: starting...
W20260811 10:16:33.802464 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:33 dist-test-slave-7f01 krb5kdc[13537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:33.970862 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:34.067476 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:34.067950 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:34.068009 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:34.069610 13561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:34.069708 13561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:34.069736 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:34.069761 13561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:34.069792 13561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:34.071897 13561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:34.072299 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:34.072634 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:34.079181 13577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:34.079181 13573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:34.079185 13575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:34.079557 13561 server_base.cc:1034] running on GCE node
I20260811 10:16:34.079980 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:34.080363 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:34.081625 13561 hybrid_clock.cc:648] HybridClock initialized: now 1786443394081594 us; error 54 us; skew 500 ppm
Aug 11 10:16:34 dist-test-slave-7f01 krb5kdc[13537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:34.085551 13561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:34.086154 13561 webserver.cc:460] Webserver started at http://127.0.0.1:37681/ using document root <none> and password file <none>
I20260811 10:16:34.086426 13561 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:34.086476 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:34.086642 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:34.087978 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "8725db5871c549499656bcc36f40158d"
format_stamp: "Formatted at 2026-08-11 10:16:34 on dist-test-slave-7f01"
server_key: "5fde183e18dbfa212bd1564936e9a5a0"
server_key_iv: "13201acc3701438dfb45ea932eb5c071"
server_key_version: "encryptionkey@0"
I20260811 10:16:34.088388 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "8725db5871c549499656bcc36f40158d"
format_stamp: "Formatted at 2026-08-11 10:16:34 on dist-test-slave-7f01"
server_key: "5fde183e18dbfa212bd1564936e9a5a0"
server_key_iv: "13201acc3701438dfb45ea932eb5c071"
server_key_version: "encryptionkey@0"
I20260811 10:16:34.089978 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:34.091063 13586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:34.091287 13561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:34.091393 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "8725db5871c549499656bcc36f40158d"
format_stamp: "Formatted at 2026-08-11 10:16:34 on dist-test-slave-7f01"
server_key: "5fde183e18dbfa212bd1564936e9a5a0"
server_key_iv: "13201acc3701438dfb45ea932eb5c071"
server_key_version: "encryptionkey@0"
I20260811 10:16:34.091463 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:34.100006 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:34.102535 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:34.102671 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:34.107149 13561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46365
I20260811 10:16:34.107246 13678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46365 every 8 connection(s)
I20260811 10:16:34.107626 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:34.108670 13680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:34.109387 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13561
I20260811 10:16:34.109498 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:34.109714 9144 external_mini_cluster.cc:1428] Setting key 75f4321432f1d00b01fb7c631cc38f8a
I20260811 10:16:34.112275 13680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: Bootstrap starting.
I20260811 10:16:34.113072 13680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:34.113423 13680 log.cc:824] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:34.114209 13680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: No bootstrap required, opened a new log
I20260811 10:16:34.116475 13680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260811 10:16:34.116667 13680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:34.116721 13680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8725db5871c549499656bcc36f40158d, State: Initialized, Role: FOLLOWER
I20260811 10:16:34.116907 13680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260811 10:16:34.117033 13680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:34.117075 13680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:34.117127 13680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:34.117414 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:34.110461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:49957 (local address 127.0.0.1:46365)
0811 10:16:34.110631 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:34.110641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:34.110676 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:16:34.111275 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:34.111284 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:34.111357 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:34.111527 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:34.111814 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:34.113214 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:34.114375 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:34.114996 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:34.115191 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:34.116392 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:34.116397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:34.116398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:34.116654 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:34.116730 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:34.116752 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:34.116869 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:34.117114 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 10:16:34.117164 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260811 10:16:34.117910 13680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260811 10:16:34.118027 13680 leader_election.cc:302] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8725db5871c549499656bcc36f40158d; no voters:
I20260811 10:16:34.118291 13680 leader_election.cc:288] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:34.118381 13689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:34.118603 13689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [term 1 LEADER]: Becoming Leader. State: Replica: 8725db5871c549499656bcc36f40158d, State: Running, Role: LEADER
I20260811 10:16:34.118649 13680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:34.118894 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:34.118816 13689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } }
I20260811 10:16:34.120117 13693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8725db5871c549499656bcc36f40158d. Latest consensus state: current_term: 1 leader_uuid: "8725db5871c549499656bcc36f40158d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } } }
I20260811 10:16:34.120249 13693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:34.120697 13690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8725db5871c549499656bcc36f40158d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8725db5871c549499656bcc36f40158d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46365 } } }
I20260811 10:16:34.121001 13690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:34.121155 13722 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:34.121243 13722 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:34.121307 13718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:34.121565 13718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:34.123327 13718 catalog_manager.cc:1347] Generated new cluster ID: 41da4a19953040a88006889a9d71380d
I20260811 10:16:34.123386 13718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:34.126204 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:34.121014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:38895 (local address 127.0.0.1:46365)
0811 10:16:34.121070 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:34.121073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:34.121554 (+ 481us) server_negotiation.cc:407] Connection header received
0811 10:16:34.121619 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:34.121620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:34.121637 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:34.121713 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:34.121922 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:34.122775 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:34.124404 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:34.124762 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:34.124814 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:34.125859 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:34.125861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:34.125861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:34.125867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:34.125888 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:34.125894 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:34.125931 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:34.126099 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 10:16:34.126116 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 10:16:34.137691 13718 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:34.138373 13718 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:34.143663 13718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8725db5871c549499656bcc36f40158d: Generated new TSK 0
I20260811 10:16:34.144014 13718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:34.150811 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13561
W20260811 10:16:34.170377 13535 connection.cc:535] client connection to 127.0.0.1:46365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46365): Connection reset by peer (error 104)
2026-08-11T10:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:34 dist-test-slave-7f01 krb5kdc[13735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:34 dist-test-slave-7f01 krb5kdc[13735](info): set up 2 sockets
Aug 11 10:16:34 dist-test-slave-7f01 krb5kdc[13735](info): commencing operation
krb5kdc: starting...
W20260811 10:16:35.479693 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:35 dist-test-slave-7f01 krb5kdc[13735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 10:16:35.636158 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:35.731256 13761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:35.731572 13761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:35.731619 13761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:35.733047 13761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:35.733139 13761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:35.733165 13761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:35.733197 13761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:35.733225 13761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:35.734941 13761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:35.735281 13761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:35.735559 13761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:35.741119 13769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:35.741169 13771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:35.741372 13761 server_base.cc:1034] running on GCE node
W20260811 10:16:35.741186 13773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:35.741757 13761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:35.742118 13761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:35.743340 13761 hybrid_clock.cc:648] HybridClock initialized: now 1786443395743252 us; error 112 us; skew 500 ppm
Aug 11 10:16:35 dist-test-slave-7f01 krb5kdc[13735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443395, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:35.751675 13761 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 10:16:35.752365 13761 webserver.cc:460] Webserver started at http://127.0.0.1:34779/ using document root <none> and password file <none>
I20260811 10:16:35.752638 13761 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:35.752714 13761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:35.752882 13761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:35.754106 13761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "963e78b750a3477c8dea6c549a82742c"
format_stamp: "Formatted at 2026-08-11 10:16:35 on dist-test-slave-7f01"
server_key: "8ef67e76ca111d5d561f02dd19d6345a"
server_key_iv: "308b1c9a09a786c1d22c378a15462358"
server_key_version: "encryptionkey@0"
I20260811 10:16:35.754547 13761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "963e78b750a3477c8dea6c549a82742c"
format_stamp: "Formatted at 2026-08-11 10:16:35 on dist-test-slave-7f01"
server_key: "8ef67e76ca111d5d561f02dd19d6345a"
server_key_iv: "308b1c9a09a786c1d22c378a15462358"
server_key_version: "encryptionkey@0"
I20260811 10:16:35.756388 13761 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 10:16:35.757404 13785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:35.757673 13761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:35.757838 13761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "963e78b750a3477c8dea6c549a82742c"
format_stamp: "Formatted at 2026-08-11 10:16:35 on dist-test-slave-7f01"
server_key: "8ef67e76ca111d5d561f02dd19d6345a"
server_key_iv: "308b1c9a09a786c1d22c378a15462358"
server_key_version: "encryptionkey@0"
I20260811 10:16:35.757943 13761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:35.761938 13761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:35.764489 13761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:35.764680 13761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:35.769663 13761 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45367
I20260811 10:16:35.769699 13873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45367 every 8 connection(s)
I20260811 10:16:35.770098 13761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:35.771283 13874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:35.773919 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: Bootstrap starting.
I20260811 10:16:35.774562 13874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:35.774921 13874 log.cc:824] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:35.774999 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13761
I20260811 10:16:35.775134 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:35.775374 9144 external_mini_cluster.cc:1428] Setting key a4dc545ce03b37777c3528f733fc1e70
I20260811 10:16:35.776458 13874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: No bootstrap required, opened a new log
I20260811 10:16:35.779064 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:35.776246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:33059 (local address 127.0.0.1:45367)
0811 10:16:35.776551 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:35.776565 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:35.776615 (+ 50us) server_negotiation.cc:407] Connection header received
0811 10:16:35.777271 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:35.777281 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:35.777314 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:35.777479 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:35.778392 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:35.778397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:35.778397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:35.778402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:35.778460 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:35.778474 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:35.778585 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:35.778675 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 10:16:35.778765 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":142,"threads_started":1}
I20260811 10:16:35.780068 13874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } }
I20260811 10:16:35.780298 13874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:35.780345 13874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 963e78b750a3477c8dea6c549a82742c, State: Initialized, Role: FOLLOWER
I20260811 10:16:35.780505 13874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } }
I20260811 10:16:35.780623 13874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:35.780665 13874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:35.780726 13874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:35.780876 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:35.781543 13874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } }
I20260811 10:16:35.781682 13874 leader_election.cc:302] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 963e78b750a3477c8dea6c549a82742c; no voters:
I20260811 10:16:35.781903 13874 leader_election.cc:288] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:35.781989 13881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:35.782226 13874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:35.782292 13881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [term 1 LEADER]: Becoming Leader. State: Replica: 963e78b750a3477c8dea6c549a82742c, State: Running, Role: LEADER
I20260811 10:16:35.782588 13881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } }
I20260811 10:16:35.783511 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "963e78b750a3477c8dea6c549a82742c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } } }
I20260811 10:16:35.783771 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:35.784003 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 963e78b750a3477c8dea6c549a82742c. Latest consensus state: current_term: 1 leader_uuid: "963e78b750a3477c8dea6c549a82742c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "963e78b750a3477c8dea6c549a82742c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45367 } } }
I20260811 10:16:35.784065 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:35.784793 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:35.782954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:36131 (local address 127.0.0.1:45367)
0811 10:16:35.783014 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:35.783017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:35.783256 (+ 239us) server_negotiation.cc:407] Connection header received
0811 10:16:35.783370 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:35.783371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:35.783378 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:35.783415 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:35.784515 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:35.784517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:35.784518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:35.784526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:35.784551 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:35.784557 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:35.784643 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:35.784726 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:16:35.784741 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 10:16:35.784987 13908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:35.785053 13908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:35.785151 13906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:35.785353 13906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:35.787209 13906 catalog_manager.cc:1347] Generated new cluster ID: f8364252d9874cfcb8a50a4555d6673b
I20260811 10:16:35.787305 13906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:35.801612 13906 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:35.801904 13906 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:35.807636 13906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 963e78b750a3477c8dea6c549a82742c: Generated new TSK 0
I20260811 10:16:35.807934 13906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:35.819255 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13761
2026-08-11T10:16:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:35 dist-test-slave-7f01 krb5kdc[13921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:35 dist-test-slave-7f01 krb5kdc[13921](info): set up 2 sockets
Aug 11 10:16:35 dist-test-slave-7f01 krb5kdc[13921](info): commencing operation
krb5kdc: starting...
W20260811 10:16:37.171435 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[13921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:37.332259 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43315
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:43315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:37.425969 13939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:37.426225 13939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:37.426252 13939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:37.427541 13939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:37.427603 13939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:37.427618 13939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:37.427631 13939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:37.427644 13939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:37.429337 13939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:43315
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:43315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:37.429651 13939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:37.429955 13939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:37.435372 13946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:37.435314 13948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:37.435400 13951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:37.435911 13939 server_base.cc:1034] running on GCE node
I20260811 10:16:37.436316 13939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:37.436789 13939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:37.438030 13939 hybrid_clock.cc:648] HybridClock initialized: now 1786443397437984 us; error 76 us; skew 500 ppm
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[13921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443397, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:37.441663 13939 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:37.442389 13939 webserver.cc:460] Webserver started at http://127.8.238.62:46331/ using document root <none> and password file <none>
I20260811 10:16:37.442797 13939 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:37.442909 13939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:37.443109 13939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:37.444758 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "83aeb8e61cb04e0c82487378ef0d08d0"
format_stamp: "Formatted at 2026-08-11 10:16:37 on dist-test-slave-7f01"
server_key: "9f6f8d69fc3d0a56b7ba327be5261392"
server_key_iv: "59ce14044e354dc51a7815b0e5d15a5f"
server_key_version: "encryptionkey@0"
I20260811 10:16:37.445283 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "83aeb8e61cb04e0c82487378ef0d08d0"
format_stamp: "Formatted at 2026-08-11 10:16:37 on dist-test-slave-7f01"
server_key: "9f6f8d69fc3d0a56b7ba327be5261392"
server_key_iv: "59ce14044e354dc51a7815b0e5d15a5f"
server_key_version: "encryptionkey@0"
I20260811 10:16:37.447063 13939 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:37.448359 13968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:37.448673 13939 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 10:16:37.448815 13939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "83aeb8e61cb04e0c82487378ef0d08d0"
format_stamp: "Formatted at 2026-08-11 10:16:37 on dist-test-slave-7f01"
server_key: "9f6f8d69fc3d0a56b7ba327be5261392"
server_key_iv: "59ce14044e354dc51a7815b0e5d15a5f"
server_key_version: "encryptionkey@0"
I20260811 10:16:37.448927 13939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:37.472929 13939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:37.476186 13939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:37.476408 13939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:37.481459 13939 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:43315
I20260811 10:16:37.481518 14055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:43315 every 8 connection(s)
I20260811 10:16:37.481945 13939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:37.483155 14059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:37.486557 14059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0: Bootstrap starting.
I20260811 10:16:37.487417 14059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:37.487704 14059 log.cc:824] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:37.488627 14059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0: No bootstrap required, opened a new log
I20260811 10:16:37.490463 14059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } }
I20260811 10:16:37.490631 14059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:37.490656 14059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83aeb8e61cb04e0c82487378ef0d08d0, State: Initialized, Role: FOLLOWER
I20260811 10:16:37.490820 14059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } }
I20260811 10:16:37.490927 14059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:37.490967 14059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:37.490973 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 13939
I20260811 10:16:37.491022 14059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:37.491098 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:37.491312 9144 external_mini_cluster.cc:1428] Setting key b545a743d617207c9d901851cf0c39b8
I20260811 10:16:37.492430 14059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } }
I20260811 10:16:37.492592 14059 leader_election.cc:302] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83aeb8e61cb04e0c82487378ef0d08d0; no voters:
I20260811 10:16:37.492794 14059 leader_election.cc:288] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:37.492995 14067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:37.493149 14059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:37.493237 14067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [term 1 LEADER]: Becoming Leader. State: Replica: 83aeb8e61cb04e0c82487378ef0d08d0, State: Running, Role: LEADER
I20260811 10:16:37.493366 14067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } }
I20260811 10:16:37.493935 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } } }
I20260811 10:16:37.494056 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:37.494305 14072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83aeb8e61cb04e0c82487378ef0d08d0. Latest consensus state: current_term: 1 leader_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83aeb8e61cb04e0c82487378ef0d08d0" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 43315 } } }
I20260811 10:16:37.495119 14072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:37.495304 14082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:37.495970 14082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:37.499121 14082 catalog_manager.cc:1347] Generated new cluster ID: b1bb6e0ef56c4505b926728302df015d
I20260811 10:16:37.499199 14082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[13921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:37.503801 14082 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:37.504570 14082 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:37.507310 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:37.492142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37075 (local address 127.8.238.62:43315)
0811 10:16:37.492449 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:37.492462 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:37.492611 (+ 149us) server_negotiation.cc:407] Connection header received
0811 10:16:37.492830 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:37.492839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:37.492911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:37.493076 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:37.494190 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:37.496446 (+ 2256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:37.497655 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:37.498262 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:37.498395 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:37.501887 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:37.501894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:37.501895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:37.501926 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:37.504853 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:37.505455 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:37.505459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:37.505460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:37.505589 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:37.505854 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:37.505855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:37.505856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:37.506187 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:37.506403 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:37.506931 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 10:16:37.507055 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":132,"threads_started":1}
I20260811 10:16:37.508416 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:37.510993 14082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83aeb8e61cb04e0c82487378ef0d08d0: Generated new TSK 0
I20260811 10:16:37.511209 14082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:37.517117 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:37.510174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49311 (local address 127.8.238.62:43315)
0811 10:16:37.510215 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:37.510217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:37.510228 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:37.510279 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:37.510280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:37.510291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:37.510352 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:37.511167 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:37.511833 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:37.512595 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:37.512844 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:37.512928 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:37.514286 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:37.514288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:37.514288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:37.514296 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:37.515850 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:37.516251 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:37.516252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:37.516252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:37.516320 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:37.516557 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:37.516557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:37.516558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:37.516692 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:37.516811 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:37.516941 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 10:16:37.517006 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 10:16:37.519412 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 13939
W20260811 10:16:37.545351 13918 connection.cc:535] client connection to 127.8.238.62:43315 recv error: Network error: failed to read from TLS socket (remote: 127.8.238.62:43315): Connection reset by peer (error 104)
2026-08-11T10:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[14110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[14110](info): set up 2 sockets
Aug 11 10:16:37 dist-test-slave-7f01 krb5kdc[14110](info): commencing operation
krb5kdc: starting...
W20260811 10:16:38.882797 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:38 dist-test-slave-7f01 krb5kdc[14110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:39.040902 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45575
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:45575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:39.132974 14126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:39.133260 14126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:39.133288 14126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:39.134577 14126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:39.134637 14126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:39.134652 14126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:39.134666 14126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:39.134680 14126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:39.136829 14126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:45575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:45575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:39.137336 14126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:39.137673 14126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:39.143970 14136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:39.143958 14134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:39.144207 14126 server_base.cc:1034] running on GCE node
W20260811 10:16:39.143958 14138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:39.144709 14126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:39.145105 14126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:39.146320 14126 hybrid_clock.cc:648] HybridClock initialized: now 1786443399146279 us; error 60 us; skew 500 ppm
Aug 11 10:16:39 dist-test-slave-7f01 krb5kdc[14110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443399, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:39.149802 14126 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:39.150393 14126 webserver.cc:460] Webserver started at http://127.8.238.62:35339/ using document root <none> and password file <none>
I20260811 10:16:39.150684 14126 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:39.150763 14126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:39.150931 14126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:39.152192 14126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "5078d081dff64bc7b7b30031b5c11c8c"
format_stamp: "Formatted at 2026-08-11 10:16:39 on dist-test-slave-7f01"
server_key: "ec67bd5f6b4b5c7517e0240afe5cb5a7"
server_key_iv: "a553e46b6395bc3445d76c7ff2fa45b2"
server_key_version: "encryptionkey@0"
I20260811 10:16:39.152591 14126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "5078d081dff64bc7b7b30031b5c11c8c"
format_stamp: "Formatted at 2026-08-11 10:16:39 on dist-test-slave-7f01"
server_key: "ec67bd5f6b4b5c7517e0240afe5cb5a7"
server_key_iv: "a553e46b6395bc3445d76c7ff2fa45b2"
server_key_version: "encryptionkey@0"
I20260811 10:16:39.154407 14126 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 10:16:39.155589 14152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:39.155952 14126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:39.156206 14126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "5078d081dff64bc7b7b30031b5c11c8c"
format_stamp: "Formatted at 2026-08-11 10:16:39 on dist-test-slave-7f01"
server_key: "ec67bd5f6b4b5c7517e0240afe5cb5a7"
server_key_iv: "a553e46b6395bc3445d76c7ff2fa45b2"
server_key_version: "encryptionkey@0"
I20260811 10:16:39.156348 14126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:39.166590 14126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:39.169389 14126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:39.169602 14126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:39.174353 14126 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:45575
I20260811 10:16:39.174580 14256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:45575 every 8 connection(s)
I20260811 10:16:39.174860 14126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:39.175911 14257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:39.178452 14257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c: Bootstrap starting.
I20260811 10:16:39.179150 14257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:39.179497 14257 log.cc:824] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:39.179947 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14126
I20260811 10:16:39.180109 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:39.180369 9144 external_mini_cluster.cc:1428] Setting key c64d97754161765f3dca0e20d4769f8d
I20260811 10:16:39.180780 14257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c: No bootstrap required, opened a new log
I20260811 10:16:39.182811 14257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } }
I20260811 10:16:39.183000 14257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:39.183027 14257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5078d081dff64bc7b7b30031b5c11c8c, State: Initialized, Role: FOLLOWER
I20260811 10:16:39.183173 14257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } }
I20260811 10:16:39.183265 14257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:39.183359 14257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:39.183432 14257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:39.184217 14257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } }
I20260811 10:16:39.184324 14257 leader_election.cc:302] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5078d081dff64bc7b7b30031b5c11c8c; no voters:
I20260811 10:16:39.184535 14257 leader_election.cc:288] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:39.184576 14267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:39.184702 14267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [term 1 LEADER]: Becoming Leader. State: Replica: 5078d081dff64bc7b7b30031b5c11c8c, State: Running, Role: LEADER
I20260811 10:16:39.184860 14257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:39.184876 14267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } }
I20260811 10:16:39.185357 14267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5078d081dff64bc7b7b30031b5c11c8c. Latest consensus state: current_term: 1 leader_uuid: "5078d081dff64bc7b7b30031b5c11c8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } } }
I20260811 10:16:39.185460 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5078d081dff64bc7b7b30031b5c11c8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5078d081dff64bc7b7b30031b5c11c8c" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 45575 } } }
I20260811 10:16:39.185500 14267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:39.185629 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:39.185907 14279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:39.186740 14279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:39.188621 14279 catalog_manager.cc:1347] Generated new cluster ID: cf71a6f798354283a90d7d3a7adf4444
I20260811 10:16:39.188702 14279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:39.189592 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:39.181130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35351 (local address 127.8.238.62:45575)
0811 10:16:39.181400 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:39.181421 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:39.181462 (+ 41us) server_negotiation.cc:407] Connection header received
0811 10:16:39.182823 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:39.182836 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:39.182918 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:39.183266 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:39.183700 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:39.185641 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:39.186918 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:39.187676 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:39.187817 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:39.188668 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:39.188672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:39.188672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:39.188926 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:39.188970 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:39.188986 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:39.189082 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:39.189203 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 10:16:39.189359 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":111,"threads_started":1}
I20260811 10:16:39.190711 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:39.196329 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:39.192492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59299 (local address 127.8.238.62:45575)
0811 10:16:39.192534 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:39.192536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:39.192551 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:16:39.192680 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:39.192680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:39.192708 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:39.192764 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:39.192930 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:39.193731 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:39.194776 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:39.195111 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:39.195192 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:39.196037 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:39.196039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:39.196039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:39.196046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:39.196064 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:39.196069 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:39.196136 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:39.196211 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 10:16:39.196226 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:16:39.201712 14279 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:39.202281 14279 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:39.209517 14279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5078d081dff64bc7b7b30031b5c11c8c: Generated new TSK 0
I20260811 10:16:39.209787 14279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:39.221887 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14126
W20260811 10:16:39.242071 14108 connection.cc:535] client connection to 127.8.238.62:45575 recv error: Network error: failed to read from TLS socket (remote: 127.8.238.62:45575): Connection reset by peer (error 104)
2026-08-11T10:16:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1696 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:39 dist-test-slave-7f01 krb5kdc[14309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:39 dist-test-slave-7f01 krb5kdc[14309](info): set up 2 sockets
Aug 11 10:16:39 dist-test-slave-7f01 krb5kdc[14309](info): commencing operation
krb5kdc: starting...
W20260811 10:16:40.591940 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.004s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:40 dist-test-slave-7f01 krb5kdc[14309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:40.751464 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46865
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:46865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:40.853333 14325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:40.853684 14325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:40.853734 14325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:40.855029 14325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:40.855118 14325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:40.855144 14325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:40.855166 14325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:40.855197 14325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:40.856973 14325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:46865
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:40.857311 14325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:40.857585 14325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:40.863497 14335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:40.863540 14336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:40.863540 14342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:40.863655 14325 server_base.cc:1034] running on GCE node
I20260811 10:16:40.864312 14325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:40.864691 14325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:40.865890 14325 hybrid_clock.cc:648] HybridClock initialized: now 1786443400865845 us; error 58 us; skew 500 ppm
Aug 11 10:16:40 dist-test-slave-7f01 krb5kdc[14309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443400, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:40.872968 14325 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:40.873632 14325 webserver.cc:460] Webserver started at http://127.8.238.62:40559/ using document root <none> and password file <none>
I20260811 10:16:40.873900 14325 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:40.873975 14325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:40.874143 14325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:40.875349 14325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "c87f9afa34814857b44a33b47840484b"
format_stamp: "Formatted at 2026-08-11 10:16:40 on dist-test-slave-7f01"
server_key: "80c9e636c78e098aad5e003e976ba206"
server_key_iv: "77420f567cf8c17bcfbcf117060115f2"
server_key_version: "encryptionkey@0"
I20260811 10:16:40.875824 14325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "c87f9afa34814857b44a33b47840484b"
format_stamp: "Formatted at 2026-08-11 10:16:40 on dist-test-slave-7f01"
server_key: "80c9e636c78e098aad5e003e976ba206"
server_key_iv: "77420f567cf8c17bcfbcf117060115f2"
server_key_version: "encryptionkey@0"
I20260811 10:16:40.877638 14325 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 10:16:40.879058 14357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:40.879388 14325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:40.879539 14325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "c87f9afa34814857b44a33b47840484b"
format_stamp: "Formatted at 2026-08-11 10:16:40 on dist-test-slave-7f01"
server_key: "80c9e636c78e098aad5e003e976ba206"
server_key_iv: "77420f567cf8c17bcfbcf117060115f2"
server_key_version: "encryptionkey@0"
I20260811 10:16:40.879640 14325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:40.883317 14325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:40.886350 14325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:40.886634 14325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:40.893213 14325 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:46865
I20260811 10:16:40.893246 14460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:46865 every 8 connection(s)
I20260811 10:16:40.893685 14325 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:40.894749 14462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:40.897511 14462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b: Bootstrap starting.
I20260811 10:16:40.898231 14462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:40.898563 14462 log.cc:824] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:40.899991 14462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b: No bootstrap required, opened a new log
I20260811 10:16:40.900167 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14325
I20260811 10:16:40.900326 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:40.900599 9144 external_mini_cluster.cc:1428] Setting key aae3cc1ceda423a087742a14bd41882c
I20260811 10:16:40.902925 14462 raft_consensus.cc:348] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } }
I20260811 10:16:40.903204 14462 raft_consensus.cc:374] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:40.903291 14462 raft_consensus.cc:729] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87f9afa34814857b44a33b47840484b, State: Initialized, Role: FOLLOWER
I20260811 10:16:40.903543 14462 consensus_queue.cc:260] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } }
I20260811 10:16:40.903723 14462 raft_consensus.cc:388] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:40.903797 14462 raft_consensus.cc:482] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:40.903851 14462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:40.905378 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:40.901455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45739 (local address 127.8.238.62:46865)
0811 10:16:40.901745 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:40.901756 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:40.901793 (+ 37us) server_negotiation.cc:407] Connection header received
0811 10:16:40.902982 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:40.902998 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:40.903051 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:40.903312 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:40.904421 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:40.904428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:40.904428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:40.904436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:40.904510 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:40.904535 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:40.904785 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:40.904910 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 10:16:40.904985 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":165,"threads_started":1}
I20260811 10:16:40.905885 14462 raft_consensus.cc:504] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } }
I20260811 10:16:40.906055 14462 leader_election.cc:302] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c87f9afa34814857b44a33b47840484b; no voters:
I20260811 10:16:40.906309 14462 leader_election.cc:288] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:40.906401 14472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:40.906582 14472 raft_consensus.cc:686] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [term 1 LEADER]: Becoming Leader. State: Replica: c87f9afa34814857b44a33b47840484b, State: Running, Role: LEADER
I20260811 10:16:40.906697 14462 sys_catalog.cc:564] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:40.906692 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:40.906742 14472 consensus_queue.cc:237] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } }
I20260811 10:16:40.907249 14473 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c87f9afa34814857b44a33b47840484b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } } }
I20260811 10:16:40.907346 14473 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:40.907519 14474 sys_catalog.cc:455] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c87f9afa34814857b44a33b47840484b. Latest consensus state: current_term: 1 leader_uuid: "c87f9afa34814857b44a33b47840484b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c87f9afa34814857b44a33b47840484b" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46865 } } }
I20260811 10:16:40.907591 14474 sys_catalog.cc:458] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:40.907989 14481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:40.908572 14481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:40.910485 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:40.908488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58783 (local address 127.8.238.62:46865)
0811 10:16:40.908528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:40.908530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:40.908997 (+ 467us) server_negotiation.cc:407] Connection header received
0811 10:16:40.909076 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:40.909077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:40.909083 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:40.909147 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:40.910186 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:40.910187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:40.910188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:40.910192 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:40.910211 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:40.910216 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:40.910313 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:40.910377 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 10:16:40.910394 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:16:40.910725 14481 catalog_manager.cc:1347] Generated new cluster ID: 4760da59b63542499cf78a6b0c5311e5
I20260811 10:16:40.910781 14481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:40.927664 14481 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:40.927922 14481 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:40.931692 14481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c87f9afa34814857b44a33b47840484b: Generated new TSK 0
I20260811 10:16:40.931933 14481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:40.943243 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14325
2026-08-11T10:16:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1729 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:41 dist-test-slave-7f01 krb5kdc[14511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:41 dist-test-slave-7f01 krb5kdc[14511](info): set up 2 sockets
Aug 11 10:16:41 dist-test-slave-7f01 krb5kdc[14511](info): commencing operation
krb5kdc: starting...
W20260811 10:16:42.294152 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:42.447934 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34773
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:44545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:34773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:42.544971 14534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:42.545265 14534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:42.545313 14534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:42.546627 14534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:42.546715 14534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:42.546741 14534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:42.546800 14534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:42.546829 14534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:42.548627 14534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:44545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:34773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:34773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:42.548962 14534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:42.549263 14534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:42.554667 14544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:42.554670 14545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:42.554667 14551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:42.554848 14534 server_base.cc:1034] running on GCE node
I20260811 10:16:42.555250 14534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:42.555579 14534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:42.556783 14534 hybrid_clock.cc:648] HybridClock initialized: now 1786443402556771 us; error 43 us; skew 500 ppm
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443402, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:42.560621 14534 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:42.561350 14534 webserver.cc:460] Webserver started at http://127.8.238.62:39639/ using document root <none> and password file <none>
I20260811 10:16:42.561635 14534 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:42.561689 14534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:42.561816 14534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:42.563201 14534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "dce6fdab983a4070b91a352689295163"
format_stamp: "Formatted at 2026-08-11 10:16:42 on dist-test-slave-7f01"
server_key: "fc47eb5cb15519646912065625eeac1f"
server_key_iv: "90ebad6e63d42e8da35db3a4a2183945"
server_key_version: "encryptionkey@0"
I20260811 10:16:42.563594 14534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "dce6fdab983a4070b91a352689295163"
format_stamp: "Formatted at 2026-08-11 10:16:42 on dist-test-slave-7f01"
server_key: "fc47eb5cb15519646912065625eeac1f"
server_key_iv: "90ebad6e63d42e8da35db3a4a2183945"
server_key_version: "encryptionkey@0"
I20260811 10:16:42.565936 14534 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 10:16:42.567143 14568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:42.567545 14534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:42.567886 14534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "dce6fdab983a4070b91a352689295163"
format_stamp: "Formatted at 2026-08-11 10:16:42 on dist-test-slave-7f01"
server_key: "fc47eb5cb15519646912065625eeac1f"
server_key_iv: "90ebad6e63d42e8da35db3a4a2183945"
server_key_version: "encryptionkey@0"
I20260811 10:16:42.568029 14534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:42.578717 14534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:42.581777 14534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:42.582036 14534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:42.588153 14534 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:34773
I20260811 10:16:42.588198 14675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:34773 every 8 connection(s)
I20260811 10:16:42.588603 14534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:42.589592 14677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:42.592206 14677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163: Bootstrap starting.
I20260811 10:16:42.592860 14677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:42.593122 14677 log.cc:824] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:42.594228 14677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163: No bootstrap required, opened a new log
I20260811 10:16:42.596755 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14534
I20260811 10:16:42.596925 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:42.596802 14677 raft_consensus.cc:348] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } }
I20260811 10:16:42.597069 14677 raft_consensus.cc:374] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:42.597137 14677 raft_consensus.cc:729] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce6fdab983a4070b91a352689295163, State: Initialized, Role: FOLLOWER
I20260811 10:16:42.597260 9144 external_mini_cluster.cc:1428] Setting key d66dc1769b7f334e43382c7c0fc48635
I20260811 10:16:42.597378 14677 consensus_queue.cc:260] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } }
I20260811 10:16:42.597481 14677 raft_consensus.cc:388] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:42.597548 14677 raft_consensus.cc:482] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:42.597608 14677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:42.599110 14677 raft_consensus.cc:504] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } }
I20260811 10:16:42.599233 14677 leader_election.cc:302] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dce6fdab983a4070b91a352689295163; no voters:
I20260811 10:16:42.599437 14677 leader_election.cc:288] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:42.599598 14686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:42.599915 14677 sys_catalog.cc:564] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:42.599993 14686 raft_consensus.cc:686] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [term 1 LEADER]: Becoming Leader. State: Replica: dce6fdab983a4070b91a352689295163, State: Running, Role: LEADER
I20260811 10:16:42.600211 14686 consensus_queue.cc:237] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } }
I20260811 10:16:42.600819 14689 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dce6fdab983a4070b91a352689295163. Latest consensus state: current_term: 1 leader_uuid: "dce6fdab983a4070b91a352689295163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } } }
I20260811 10:16:42.600919 14689 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:42.601035 14688 sys_catalog.cc:455] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dce6fdab983a4070b91a352689295163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dce6fdab983a4070b91a352689295163" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 34773 } } }
I20260811 10:16:42.601104 14688 sys_catalog.cc:458] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:42.601337 14699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:42.601981 14699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:42.605245 14699 catalog_manager.cc:1347] Generated new cluster ID: 7d183385cf30479f8512cc5b098b19ed
I20260811 10:16:42.605326 14699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:42.613070 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:42.597974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47245 (local address 127.8.238.62:34773)
0811 10:16:42.598294 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:42.598307 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:42.598351 (+ 44us) server_negotiation.cc:407] Connection header received
0811 10:16:42.598528 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:42.598538 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:42.598621 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:42.598825 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:42.599967 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:42.601586 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:42.602828 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:42.603580 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:42.603723 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:42.608044 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:42.608052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:42.608053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:42.608073 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:42.610493 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:42.611006 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:42.611011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:42.611011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:42.611177 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:42.611444 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:42.611445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:42.611445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:42.611855 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:42.612025 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:42.612643 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 10:16:42.612816 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":199,"threads_started":1}
I20260811 10:16:42.614393 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:42.628221 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:42.620178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54933 (local address 127.8.238.62:34773)
0811 10:16:42.620231 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:42.620234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:42.620297 (+ 63us) server_negotiation.cc:407] Connection header received
0811 10:16:42.620395 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:42.620396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:42.620410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:42.620469 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:42.621235 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:42.622065 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:42.623139 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:42.623510 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:42.623588 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:42.625154 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:42.625156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:42.625157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:42.625165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:42.626907 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:42.627309 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:42.627311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:42.627311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:42.627389 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:42.627599 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:42.627600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:42.627600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:42.627780 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:42.627904 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:42.628006 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 10:16:42.628100 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:16:42.639681 14699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:42.640501 14699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:42.644853 14699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dce6fdab983a4070b91a352689295163: Generated new TSK 0
I20260811 10:16:42.645117 14699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:42.650704 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14534
2026-08-11T10:16:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1695 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14732](info): set up 2 sockets
Aug 11 10:16:42 dist-test-slave-7f01 krb5kdc[14732](info): commencing operation
krb5kdc: starting...
W20260811 10:16:43.995556 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:44 dist-test-slave-7f01 krb5kdc[14732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:44.151621 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35489
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:44.247786 14753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:44.248072 14753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:44.248119 14753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:44.249394 14753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:44.249442 14753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:44.249464 14753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:44.249476 14753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:44.249529 14753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:44.251092 14753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:44.251389 14753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:44.251672 14753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:44.257323 14760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:44.257536 14753 server_base.cc:1034] running on GCE node
W20260811 10:16:44.257323 14763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:44.257337 14771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:44.258352 14753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:44.258812 14753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:44.260032 14753 hybrid_clock.cc:648] HybridClock initialized: now 1786443404260010 us; error 80 us; skew 500 ppm
Aug 11 10:16:44 dist-test-slave-7f01 krb5kdc[14732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:44.263620 14753 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:44.264353 14753 webserver.cc:460] Webserver started at http://127.8.238.62:35171/ using document root <none> and password file <none>
I20260811 10:16:44.264665 14753 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:44.264719 14753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:44.264894 14753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:44.266120 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "3447cac004dc4ddd9d30f11650306235"
format_stamp: "Formatted at 2026-08-11 10:16:44 on dist-test-slave-7f01"
server_key: "3dfa8ffa02eeee20ee43b35bef9cc6c1"
server_key_iv: "f01774c43c8e863c870555c58ea4e33d"
server_key_version: "encryptionkey@0"
I20260811 10:16:44.266518 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "3447cac004dc4ddd9d30f11650306235"
format_stamp: "Formatted at 2026-08-11 10:16:44 on dist-test-slave-7f01"
server_key: "3dfa8ffa02eeee20ee43b35bef9cc6c1"
server_key_iv: "f01774c43c8e863c870555c58ea4e33d"
server_key_version: "encryptionkey@0"
I20260811 10:16:44.268196 14753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 10:16:44.269590 14783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:44.269845 14753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:44.269971 14753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "3447cac004dc4ddd9d30f11650306235"
format_stamp: "Formatted at 2026-08-11 10:16:44 on dist-test-slave-7f01"
server_key: "3dfa8ffa02eeee20ee43b35bef9cc6c1"
server_key_iv: "f01774c43c8e863c870555c58ea4e33d"
server_key_version: "encryptionkey@0"
I20260811 10:16:44.270059 14753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:44.299219 14753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:44.302532 14753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:44.302793 14753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:44.309612 14753 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:35489
I20260811 10:16:44.309659 14872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:35489 every 8 connection(s)
I20260811 10:16:44.310097 14753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:44.311208 14876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:44.314065 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235: Bootstrap starting.
I20260811 10:16:44.314812 14876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:44.315181 14876 log.cc:824] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:44.316432 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235: No bootstrap required, opened a new log
I20260811 10:16:44.318593 14876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } }
I20260811 10:16:44.318826 14876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:44.318883 14876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3447cac004dc4ddd9d30f11650306235, State: Initialized, Role: FOLLOWER
I20260811 10:16:44.319108 14876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } }
I20260811 10:16:44.319213 14876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:44.319264 14876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:44.319381 14876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:44.319592 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14753
I20260811 10:16:44.319720 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:44.319967 9144 external_mini_cluster.cc:1428] Setting key 17d0a5d028c4c40ac4699971c5b6eceb
I20260811 10:16:44.321069 14876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } }
I20260811 10:16:44.321226 14876 leader_election.cc:302] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3447cac004dc4ddd9d30f11650306235; no voters:
I20260811 10:16:44.321494 14876 leader_election.cc:288] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:44.321563 14887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:44.321843 14887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [term 1 LEADER]: Becoming Leader. State: Replica: 3447cac004dc4ddd9d30f11650306235, State: Running, Role: LEADER
I20260811 10:16:44.321940 14876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:44.322021 14887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } }
I20260811 10:16:44.322652 14889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3447cac004dc4ddd9d30f11650306235. Latest consensus state: current_term: 1 leader_uuid: "3447cac004dc4ddd9d30f11650306235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } } }
I20260811 10:16:44.322788 14889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:44.323189 14888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3447cac004dc4ddd9d30f11650306235" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3447cac004dc4ddd9d30f11650306235" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 35489 } } }
I20260811 10:16:44.323242 14888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235 [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:44.323889 14904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:44.325136 14904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:44.327397 14904 catalog_manager.cc:1347] Generated new cluster ID: e84d0d250c3049b8a0f56b1a95868749
I20260811 10:16:44.327481 14904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:44.328418 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:44.320726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39983 (local address 127.8.238.62:35489)
0811 10:16:44.320980 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:44.320990 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:44.321028 (+ 38us) server_negotiation.cc:407] Connection header received
0811 10:16:44.321167 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:44.321175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:44.321265 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:44.321502 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:44.321835 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:44.323553 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:44.325077 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:44.325940 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:44.326117 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:44.327481 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:44.327486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:44.327486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:44.327722 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:44.327793 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:44.327821 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:44.327907 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:44.328059 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 10:16:44.328145 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":135,"threads_started":1}
I20260811 10:16:44.329581 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:44.340164 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:44.331687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57825 (local address 127.8.238.62:35489)
0811 10:16:44.331732 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:44.331767 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:44.336095 (+ 4328us) server_negotiation.cc:407] Connection header received
0811 10:16:44.336151 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:44.336153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:44.336175 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:44.336292 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:44.336495 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:44.337320 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:44.338442 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:44.338793 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:44.338891 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:44.339833 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:44.339835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:44.339835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:44.339844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:44.339873 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:44.339879 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:44.339952 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:44.340021 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 10:16:44.340060 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:16:44.355305 14904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:44.356005 14904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:44.361668 14904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3447cac004dc4ddd9d30f11650306235: Generated new TSK 0
I20260811 10:16:44.362075 14904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:44.365311 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14753
2026-08-11T10:16:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1713 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:44 dist-test-slave-7f01 krb5kdc[14937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:44 dist-test-slave-7f01 krb5kdc[14937](info): set up 2 sockets
Aug 11 10:16:44 dist-test-slave-7f01 krb5kdc[14937](info): commencing operation
krb5kdc: starting...
W20260811 10:16:45.687387 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:45 dist-test-slave-7f01 krb5kdc[14937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:45.848438 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46035
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:46035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:45.948441 14955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:45.948772 14955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:45.948824 14955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:45.950713 14955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:45.950812 14955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:45.950838 14955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:45.950882 14955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:45.950915 14955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:45.953140 14955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:46035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:46035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:45.953504 14955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:45.953781 14955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:45.959692 14964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:45.959934 14955 server_base.cc:1034] running on GCE node
W20260811 10:16:45.959671 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:45.959669 14969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:45.960382 14955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:45.960836 14955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:45.962071 14955 hybrid_clock.cc:648] HybridClock initialized: now 1786443405962012 us; error 87 us; skew 500 ppm
Aug 11 10:16:45 dist-test-slave-7f01 krb5kdc[14937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443405, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:45.970149 14955 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:45.970834 14955 webserver.cc:460] Webserver started at http://127.8.238.62:40119/ using document root <none> and password file <none>
I20260811 10:16:45.971114 14955 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:45.971194 14955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:45.971364 14955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:45.972678 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "1fca37ccb5c24665b2e01944bad8f8fe"
format_stamp: "Formatted at 2026-08-11 10:16:45 on dist-test-slave-7f01"
server_key: "d733c01979d3df404b4ee6c2a91245bc"
server_key_iv: "56408dca38c52c00c0ec5413fb5d62e9"
server_key_version: "encryptionkey@0"
I20260811 10:16:45.973106 14955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "1fca37ccb5c24665b2e01944bad8f8fe"
format_stamp: "Formatted at 2026-08-11 10:16:45 on dist-test-slave-7f01"
server_key: "d733c01979d3df404b4ee6c2a91245bc"
server_key_iv: "56408dca38c52c00c0ec5413fb5d62e9"
server_key_version: "encryptionkey@0"
I20260811 10:16:45.974658 14955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:16:45.975898 14979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:45.976171 14955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:45.976311 14955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "1fca37ccb5c24665b2e01944bad8f8fe"
format_stamp: "Formatted at 2026-08-11 10:16:45 on dist-test-slave-7f01"
server_key: "d733c01979d3df404b4ee6c2a91245bc"
server_key_iv: "56408dca38c52c00c0ec5413fb5d62e9"
server_key_version: "encryptionkey@0"
I20260811 10:16:45.976413 14955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:45.980345 14955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:45.982978 14955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:45.983166 14955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:45.988718 14955 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:46035
I20260811 10:16:45.988762 15053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:46035 every 8 connection(s)
I20260811 10:16:45.989161 14955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:45.990739 15054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:45.994165 15054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: Bootstrap starting.
I20260811 10:16:45.995023 15054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:45.995357 15054 log.cc:824] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:45.996816 15054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: No bootstrap required, opened a new log
I20260811 10:16:45.997649 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 14955
I20260811 10:16:45.997793 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:45.997991 9144 external_mini_cluster.cc:1428] Setting key fd19ea3353f9f56a6164cce883386f96
I20260811 10:16:45.999507 15054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } }
I20260811 10:16:45.999725 15054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 10:16:45.999857 15054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fca37ccb5c24665b2e01944bad8f8fe, State: Initialized, Role: FOLLOWER
I20260811 10:16:46.000025 15054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } }
I20260811 10:16:46.000123 15054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 10:16:46.000149 15054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 10:16:46.000229 15054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:46.000999 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:45.998643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37509 (local address 127.8.238.62:46035)
0811 10:16:45.998891 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:45.998901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:45.998931 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:16:45.999491 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:45.999501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:45.999536 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:45.999677 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:46.000580 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:46.000584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:46.000584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:46.000587 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:46.000623 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:46.000635 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:46.000695 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:46.000787 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 10:16:46.000824 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
I20260811 10:16:46.001820 15054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } }
I20260811 10:16:46.001974 15054 leader_election.cc:302] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fca37ccb5c24665b2e01944bad8f8fe; no voters:
I20260811 10:16:46.002184 15054 leader_election.cc:288] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 10:16:46.002308 15060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:46.002535 15054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:46.002584 15060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [term 1 LEADER]: Becoming Leader. State: Replica: 1fca37ccb5c24665b2e01944bad8f8fe, State: Running, Role: LEADER
I20260811 10:16:46.002635 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:46.002842 15060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } }
I20260811 10:16:46.003468 15064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fca37ccb5c24665b2e01944bad8f8fe. Latest consensus state: current_term: 1 leader_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } } }
I20260811 10:16:46.003618 15064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:46.003465 15062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fca37ccb5c24665b2e01944bad8f8fe" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 46035 } } }
I20260811 10:16:46.004014 15062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe [sys.catalog]: This master's current role is: LEADER
W20260811 10:16:46.004895 15098 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:46.004999 15098 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:46.005091 15086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:46.005303 15086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:46.006264 15059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:46.004905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44587 (local address 127.8.238.62:46035)
0811 10:16:46.004943 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:46.004946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:46.005067 (+ 121us) server_negotiation.cc:407] Connection header received
0811 10:16:46.005184 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:46.005185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:46.005193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:46.005251 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:46.005993 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:46.005994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:46.005995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 10:16:46.005999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:46.006018 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 10:16:46.006022 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:46.006088 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:46.006157 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 10:16:46.006175 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 10:16:46.007210 15086 catalog_manager.cc:1347] Generated new cluster ID: 5c8aa50512a340448f1957e7bfdfd020
I20260811 10:16:46.007266 15086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:46.024302 15086 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:46.024544 15086 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:46.032140 15086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fca37ccb5c24665b2e01944bad8f8fe: Generated new TSK 0
I20260811 10:16:46.032366 15086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 10:16:46.040042 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 14955
2026-08-11T10:16:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1676 ms)
[----------] 18 tests from AuthTokenIssuingTest (30829 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:46 dist-test-slave-7f01 krb5kdc[15116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:46 dist-test-slave-7f01 krb5kdc[15116](info): set up 2 sockets
Aug 11 10:16:46 dist-test-slave-7f01 krb5kdc[15116](info): commencing operation
krb5kdc: starting...
W20260811 10:16:47.385375 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:47.540972 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42377
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:47.630451 15139 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:47.630712 15139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:47.630731 15139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:47.630755 15139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:47.631947 15139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:47.631991 15139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:47.632005 15139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:47.632018 15139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:47.632030 15139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:47.633548 15139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:42377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:47.633776 15139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:47.633980 15139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:47.639710 15145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:47.639701 15144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:47.639701 15147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:47.640074 15139 server_base.cc:1034] running on GCE node
I20260811 10:16:47.640429 15139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:47.640887 15139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:47.642165 15139 hybrid_clock.cc:648] HybridClock initialized: now 1786443407642142 us; error 47 us; skew 500 ppm
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:47.645833 15139 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:47.646406 15139 webserver.cc:460] Webserver started at http://127.8.238.62:37795/ using document root <none> and password file <none>
I20260811 10:16:47.646739 15139 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:47.646811 15139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:47.646975 15139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:47.648366 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "fa42b32b224646d9babf99a92186571e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "fdb1ff35444d34aabfbdab6499bd8f82"
server_key_iv: "0c9e29cbd5e8ae8e6175b598ded69eef"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.648818 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "fa42b32b224646d9babf99a92186571e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "fdb1ff35444d34aabfbdab6499bd8f82"
server_key_iv: "0c9e29cbd5e8ae8e6175b598ded69eef"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.650882 15139 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:47.651988 15159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:47.652282 15139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:47.652403 15139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "fa42b32b224646d9babf99a92186571e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "fdb1ff35444d34aabfbdab6499bd8f82"
server_key_iv: "0c9e29cbd5e8ae8e6175b598ded69eef"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.652509 15139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:47.667344 15139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:47.669934 15139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:47.670105 15139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:47.674729 15139 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:42377
I20260811 10:16:47.674805 15251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:42377 every 8 connection(s)
I20260811 10:16:47.675278 15139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:47.676431 15253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:47.678372 15253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:47.679404 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15139
I20260811 10:16:47.679541 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:47.679788 9144 external_mini_cluster.cc:1428] Setting key d79bd51f6e671e809597814eb397a5a8
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.61@KRBTEST.COM; defaulting to no policy
I20260811 10:16:47.692111 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.679245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42377 (local address 127.0.0.1:58716)
0811 10:16:47.679571 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.679671 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:47.680579 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:47.681039 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:47.681044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:47.681375 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:47.681556 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.681562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.682764 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.682765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.684007 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.684011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.684625 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.684626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.684722 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.685500 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:47.685511 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:47.687486 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:47.690132 (+ 2646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.690143 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.690146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.690454 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.690784 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.690785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.690785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.690892 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.691347 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:47.691353 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:47.691435 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 10:16:47.691551 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 10:16:47.691813 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260811 10:16:47.692147 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.679449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.8.238.62:42377)
0811 10:16:47.679784 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:47.679790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:47.679934 (+ 144us) server_negotiation.cc:407] Connection header received
0811 10:16:47.680807 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:47.680811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:47.680843 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:47.680987 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:47.681689 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.682639 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.684146 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.684487 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.684583 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.687621 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:47.687631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:47.687631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:47.687640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:47.689953 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.690591 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.690595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.690595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.690698 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.690975 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.690976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.690976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.691255 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:47.691375 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:47.691635 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 10:16:47.691874 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":260,"thread_start_us":188,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.61@KRBTEST.COM" created.
I20260811 10:16:47.693320 15253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:47.693770 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.693629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48266)
0811 10:16:47.693686 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.693737 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 10:16:47.693974 15160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
I20260811 10:16:47.694716 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.694518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48280)
0811 10:16:47.694618 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.694659 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 10:16:47.694954 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:38955: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
I20260811 10:16:47.711951 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:38955
--webserver_interface=127.8.238.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:16:47.744271 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } attempt: 1
I20260811 10:16:47.744858 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.744696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48292)
0811 10:16:47.744772 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.744817 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 10:16:47.745237 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.745141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48308)
0811 10:16:47.745187 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.745214 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 10:16:47.745338 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:38955: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
W20260811 10:16:47.804981 15263 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:47.805348 15263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:47.805388 15263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:47.805430 15263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:47.807885 15263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:47.807979 15263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:47.808007 15263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:47.808035 15263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:47.808058 15263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:47.811064 15263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:38955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.238.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:47.811436 15263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:47.811784 15263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:16:47.813491 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } attempt: 2
I20260811 10:16:47.815622 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.815441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48314)
0811 10:16:47.815517 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.815571 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 10:16:47.816123 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.816007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48318)
0811 10:16:47.816054 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.816090 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260811 10:16:47.816255 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:38955: Network error: Client connection negotiation failed: client connection to 127.8.238.61:38955: connect: Connection refused (error 111)
W20260811 10:16:47.819614 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:47.819615 15270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:47.819654 15277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:47.819940 15263 server_base.cc:1034] running on GCE node
I20260811 10:16:47.820312 15263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:47.820695 15263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:47.821890 15263 hybrid_clock.cc:648] HybridClock initialized: now 1786443407821867 us; error 43 us; skew 500 ppm
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:47.825781 15263 init.cc:377] Logged in from keytab as kudu/127.8.238.61@KRBTEST.COM (short username kudu)
I20260811 10:16:47.826411 15263 webserver.cc:460] Webserver started at http://127.8.238.61:43869/ using document root <none> and password file <none>
I20260811 10:16:47.826715 15263 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:47.826807 15263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:47.826975 15263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:47.828298 15263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data/instance:
uuid: "d8ba0e2a364a43e7af501a5bd9fa507e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "5a25990386f47e689a2fff03167d1e99"
server_key_iv: "c9364b83955a156d585c0d590e496ee1"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.828728 15263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal/instance:
uuid: "d8ba0e2a364a43e7af501a5bd9fa507e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "5a25990386f47e689a2fff03167d1e99"
server_key_iv: "c9364b83955a156d585c0d590e496ee1"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.830435 15263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:47.831367 15285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:47.831897 15263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:47.832019 15263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "d8ba0e2a364a43e7af501a5bd9fa507e"
format_stamp: "Formatted at 2026-08-11 10:16:47 on dist-test-slave-7f01"
server_key: "5a25990386f47e689a2fff03167d1e99"
server_key_iv: "c9364b83955a156d585c0d590e496ee1"
server_key_version: "encryptionkey@0"
I20260811 10:16:47.832118 15263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:47.840718 15263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:47.843958 15263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:47.844174 15263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:47.850320 15263 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.61:38955
I20260811 10:16:47.850426 15374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.61:38955 every 8 connection(s)
I20260811 10:16:47.850832 15263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
I20260811 10:16:47.852065 15376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:47.853773 15376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:47.860767 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15263
I20260811 10:16:47.860927 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-1/wal/instance
I20260811 10:16:47.861195 9144 external_mini_cluster.cc:1428] Setting key 700fb329acde5442b005d5293c5734b3
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:47.866405 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.854734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.8.238.62:42377)
0811 10:16:47.854854 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:47.854858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:47.855207 (+ 349us) server_negotiation.cc:407] Connection header received
0811 10:16:47.855905 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:47.855906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:47.855925 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:47.856002 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:47.856809 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.857368 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.858755 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.859008 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.859088 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.862386 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:47.862389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:47.862390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:47.862401 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:47.864577 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.865221 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.865225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.865225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.865297 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.865597 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.865598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.865598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.865742 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:47.865827 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:47.866179 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 10:16:47.866280 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 10:16:47.866657 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.854672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42377 (local address 127.0.0.1:58724)
0811 10:16:47.854996 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.855046 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:47.855654 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:47.856088 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:47.856092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:47.856394 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:47.856669 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.856674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.857523 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.857524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.858620 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.858625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.859160 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.859161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.859340 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.860167 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:47.860178 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:47.862229 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:47.864723 (+ 2494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.864743 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.864747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.865114 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.865384 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.865385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.865386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.865522 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.865819 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:47.865828 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:47.866029 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 10:16:47.866158 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 10:16:47.866372 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":166,"threads_started":1}
I20260811 10:16:47.867499 15376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.238.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.60@KRBTEST.COM" created.
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:47.881546 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.867892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48332)
0811 10:16:47.867950 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.867961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:47.868010 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:47.868583 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:47.868584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:47.868755 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:47.868819 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.868821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.869756 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.869757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.870534 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.870543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.871237 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.871238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.871322 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.872029 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:47.872034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:47.876651 (+ 4617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:47.880135 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.880135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.880136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.880395 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.880614 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.880614 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.880615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.880671 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.881109 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:47.881110 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:47.881156 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 10:16:47.881220 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 10:16:47.881372 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:16:47.882277 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.868026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.8.238.61:38955)
0811 10:16:47.868375 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:47.868380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:47.868405 (+ 25us) server_negotiation.cc:407] Connection header received
0811 10:16:47.868429 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:47.868431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:47.868447 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:47.868578 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:47.868912 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.869660 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.870700 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.871131 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.871229 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.876798 (+ 5569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:47.876862 (+ 64us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:47.876862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:47.876872 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:47.879970 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.880484 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.880489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.880489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.880551 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.880730 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.880730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.880731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.880978 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:47.881083 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:47.882069 (+ 986us) server_negotiation.cc:299] Negotiation successful
0811 10:16:47.882173 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":232,"threads_started":1}
I20260811 10:16:47.883524 15376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:47.884045 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.883886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44380)
0811 10:16:47.883936 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.884008 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 10:16:47.886132 15286 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
I20260811 10:16:47.886988 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.886833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44392)
0811 10:16:47.886887 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.886943 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 10:16:47.887147 15376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
I20260811 10:16:47.900887 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:36139
--webserver_interface=127.8.238.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:16:47.907455 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } attempt: 3
Aug 11 10:16:47 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:47.916613 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.908165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.8.238.61:38955)
0811 10:16:47.908244 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:47.908246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:47.908258 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:47.908381 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:47.908383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:47.908396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:47.908472 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:47.908845 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.909386 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.910118 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.910386 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.910482 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.913081 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:47.913083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:47.913083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:47.913092 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:47.914941 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.915515 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.915517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.915518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.915611 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:47.915914 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:47.915915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:47.915915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:47.916088 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:47.916176 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:47.916376 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 10:16:47.916485 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 10:16:47.916613 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.908018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48342)
0811 10:16:47.908155 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.908177 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:47.908274 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:47.908478 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:47.908479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:47.908676 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:47.908749 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.908751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.909479 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.909480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.910023 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:47.910025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:47.910482 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:47.910484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:47.910575 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:47.911199 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:47.911211 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:47.912942 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:47.915145 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.915147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.915148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.915420 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.915694 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:47.915695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:47.915696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:47.915829 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:47.916179 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:47.916181 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:47.916253 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 10:16:47.916330 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 10:16:47.916480 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 10:16:47.917376 15253 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:47.917979 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.917762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44408)
0811 10:16:47.917866 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.917928 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 10:16:47.918460 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.918354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44424)
0811 10:16:47.918404 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.918434 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 10:16:47.918576 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
I20260811 10:16:47.936416 15376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 1
I20260811 10:16:47.937150 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.936949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44432)
0811 10:16:47.937037 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.937105 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 10:16:47.937567 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.937454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44436)
0811 10:16:47.937504 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.937548 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 10:16:47.937669 15376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
I20260811 10:16:47.949751 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 1
I20260811 10:16:47.950452 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.950279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44438)
0811 10:16:47.950372 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.950417 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 10:16:47.950842 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:47.950742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44444)
0811 10:16:47.950788 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:47.950817 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 10:16:47.950935 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
W20260811 10:16:48.003189 15389 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:48.003469 15389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:48.003489 15389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:48.003512 15389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:48.004828 15389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:48.004889 15389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:48.004902 15389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:48.004916 15389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:48.004928 15389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:48.005841 15376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 2
I20260811 10:16:48.006444 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.006265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44446)
0811 10:16:48.006360 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.006409 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 10:16:48.006835 15389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:42377,127.8.238.61:38955,127.8.238.60:36139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:36139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.238.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:48.006873 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.006745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44448)
0811 10:16:48.006811 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.006842 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 10:16:48.006968 15376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
I20260811 10:16:48.007166 15389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:48.007437 15389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:48.012814 15399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:48.012814 15396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:48.012894 15395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:48.013332 15389 server_base.cc:1034] running on GCE node
I20260811 10:16:48.013667 15389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:48.014032 15389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:48.015239 15389 hybrid_clock.cc:648] HybridClock initialized: now 1786443408015224 us; error 38 us; skew 500 ppm
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443408, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:48.018859 15389 init.cc:377] Logged in from keytab as kudu/127.8.238.60@KRBTEST.COM (short username kudu)
I20260811 10:16:48.019575 15389 webserver.cc:460] Webserver started at http://127.8.238.60:33697/ using document root <none> and password file <none>
I20260811 10:16:48.019910 15389 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:48.019964 15389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:48.020095 15389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:48.021380 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data/instance:
uuid: "944b8a1b04204e6cbc0da4addc4a5e27"
format_stamp: "Formatted at 2026-08-11 10:16:48 on dist-test-slave-7f01"
server_key: "442dc5fe08833c4ae0165bdfd66b992e"
server_key_iv: "b55abbbd70d2e7cde0d73d73c85accbe"
server_key_version: "encryptionkey@0"
I20260811 10:16:48.021783 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal/instance:
uuid: "944b8a1b04204e6cbc0da4addc4a5e27"
format_stamp: "Formatted at 2026-08-11 10:16:48 on dist-test-slave-7f01"
server_key: "442dc5fe08833c4ae0165bdfd66b992e"
server_key_iv: "b55abbbd70d2e7cde0d73d73c85accbe"
server_key_version: "encryptionkey@0"
I20260811 10:16:48.023638 15389 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 10:16:48.024931 15406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:48.025233 15389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:48.025359 15389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "944b8a1b04204e6cbc0da4addc4a5e27"
format_stamp: "Formatted at 2026-08-11 10:16:48 on dist-test-slave-7f01"
server_key: "442dc5fe08833c4ae0165bdfd66b992e"
server_key_iv: "b55abbbd70d2e7cde0d73d73c85accbe"
server_key_version: "encryptionkey@0"
I20260811 10:16:48.025461 15389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:48.026099 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 2
I20260811 10:16:48.026655 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.026502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44454)
0811 10:16:48.026569 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.026620 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 10:16:48.027096 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.026991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44456)
0811 10:16:48.027043 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.027074 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 10:16:48.027205 15253 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:36139: Network error: Client connection negotiation failed: client connection to 127.8.238.60:36139: connect: Connection refused (error 111)
I20260811 10:16:48.048494 15389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:48.051574 15389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:48.051805 15389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:48.056720 15389 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.60:36139
I20260811 10:16:48.056771 15494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.60:36139 every 8 connection(s)
I20260811 10:16:48.057216 15389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
I20260811 10:16:48.058373 15496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:48.060089 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15389
I20260811 10:16:48.060225 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786443124069174-9144-0/minicluster-data/master-2/wal/instance
I20260811 10:16:48.060442 9144 external_mini_cluster.cc:1428] Setting key 6e07efd422a91660ca3c71f5fc41b304
I20260811 10:16:48.060484 15496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443408, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:16:48.073873 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.061602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.8.238.62:42377)
0811 10:16:48.061660 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.061662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.061834 (+ 172us) server_negotiation.cc:407] Connection header received
0811 10:16:48.062464 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.062465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.062477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.062544 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.063585 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.064556 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.066247 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.066581 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.066672 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.069986 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.069990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.069991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.070004 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.072429 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.072901 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.072903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.072903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.072971 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.073217 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.073218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.073218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.073368 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.073448 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.073663 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.073758 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:16:48.074122 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.061366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:42377 (local address 127.0.0.1:58740)
0811 10:16:48.061643 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.061724 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:48.062327 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:48.062615 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:48.062619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:48.063099 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:48.063437 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.063445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.064718 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.064720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.066117 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.066121 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.066724 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.066726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.066892 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.067695 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:48.067707 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:48.069866 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:48.072551 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.072559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.072563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.072796 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.073042 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.073043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.073043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.073145 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.073439 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:48.073447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:48.073533 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 10:16:48.073640 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 10:16:48.073844 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":92,"threads_started":1}
I20260811 10:16:48.074574 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.061307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.8.238.60:36139)
0811 10:16:48.061674 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.061685 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.061720 (+ 35us) server_negotiation.cc:407] Connection header received
0811 10:16:48.062287 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.062296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.062322 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.062485 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.063479 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.064949 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.066255 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.066808 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.066891 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.070537 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.070559 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.070559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.070569 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.072427 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.072870 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.072874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.072874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.072974 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.073202 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.073203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.073204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.073465 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.073634 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.073727 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.074427 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":315,"thread_start_us":134,"threads_started":1}
I20260811 10:16:48.074765 15496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443408, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:48.084301 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.075573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.8.238.62:42377)
0811 10:16:48.075634 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.075636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.075644 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:16:48.075662 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.075663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.075672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.075743 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.076272 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.076739 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.077431 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.077639 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.077714 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.080289 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.080290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.080291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.080298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.082246 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.082771 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.082772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.082773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.082850 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.083124 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.083125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.083126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.083341 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.083602 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.083851 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.084006 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:16:48.084780 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.075142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.8.238.61:38955)
0811 10:16:48.075214 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.075217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.075225 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:16:48.075252 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.075253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.075263 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.075332 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.075680 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.076227 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.076924 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.077160 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.077250 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.079460 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.079463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.079463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.079484 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.082652 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.083331 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.083333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.083333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.083415 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.083812 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.083815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.083816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.084193 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.084320 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.084609 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.084713 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 10:16:48.084868 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.075048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:38955 (local address 127.0.0.1:48356)
0811 10:16:48.075104 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.075116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:48.075158 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:48.075336 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:48.075337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:48.075542 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:48.075612 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.075614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.076313 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.076314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.076831 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.076832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.077251 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.077252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.077324 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.077820 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:48.077826 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:48.079272 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:48.082970 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.082972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.082973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.083231 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.083510 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.083511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.083512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.083606 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.084351 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:48.084354 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:48.084453 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 10:16:48.084572 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 10:16:48.084729 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:16:48.085531 15496 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443408, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:48.094290 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.086013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.8.238.60:36139)
0811 10:16:48.086124 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.086127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.086134 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:16:48.086154 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.086155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.086164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.086245 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.086584 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.087106 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.088009 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.088233 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.088322 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.090434 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.090436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.090436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.090443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.092394 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.092962 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.092963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.092963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.093119 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.093434 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.093435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.093435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.093690 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.093767 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.094012 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.094167 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 10:16:48.094707 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.085941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44466)
0811 10:16:48.085981 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.085993 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:48.086060 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:48.086248 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:48.086249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:48.086426 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:48.086498 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.086500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.087221 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.087223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.087906 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.087908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.088320 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.088321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.088367 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.088962 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:48.088967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:48.090331 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:48.092514 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.092515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.092516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.092848 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.093246 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.093248 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.093248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.093320 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.093792 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:48.093794 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:48.093872 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 10:16:48.093953 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 10:16:48.094610 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":78}
I20260811 10:16:48.096083 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.085547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.8.238.61:38955)
0811 10:16:48.085606 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.085608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.085618 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:48.085644 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.085644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.085657 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.085730 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.086481 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.087264 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.088033 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.088272 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.088370 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.091138 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.091140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.091141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.091150 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.093569 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.094015 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.094016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.094017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.094136 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.094935 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.094936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.094937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.095126 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.095412 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.095907 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.096015 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 10:16:48.096859 15496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: Bootstrap starting.
I20260811 10:16:48.097831 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:48.098100 15496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:48.098097 15376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 3
I20260811 10:16:48.098507 15496 log.cc:824] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:48.099488 15496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: No bootstrap required, opened a new log
I20260811 10:16:48.099032 15206 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fa42b32b224646d9babf99a92186571e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58746
I20260811 10:16:48.101670 15496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.101852 15496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944b8a1b04204e6cbc0da4addc4a5e27, State: Initialized, Role: FOLLOWER
I20260811 10:16:48.101969 15496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.102396 15496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:48.102345 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.102527 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
W20260811 10:16:48.104496 15530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:48.104611 15530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:48.107173 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.098544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.8.238.60:36139)
0811 10:16:48.098587 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.098589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.098596 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:16:48.098620 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.098621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.098632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.098696 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.099070 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.099815 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.100789 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.101076 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.101179 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.103587 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.103589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.103589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.103597 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.105699 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.106218 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.106219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.106220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.106289 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.106526 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.106527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.106527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.106712 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.106807 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.106990 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.107081 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 10:16:48.107175 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.098360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44480)
0811 10:16:48.098416 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.098428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:48.098466 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:48.098712 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:48.098714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:48.098907 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:48.098964 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.098966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.099917 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.099918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.100681 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.100684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.101189 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.101190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.101243 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.101752 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:48.101758 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:48.103466 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:48.105815 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.105816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.105817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.106120 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.106372 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.106373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.106373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.106449 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.106801 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:48.106802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:48.106866 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 10:16:48.106942 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 10:16:48.107071 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:16:48.104856 15389 master_runner.cc:186] Error getting master registration for 127.8.238.62:42377: OK, instance_id { permanent_uuid: "fa42b32b224646d9babf99a92186571e" instance_seqno: 1786443407673445 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:16:48.108058 15389 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:16:48.109522 15376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: Bootstrap starting.
I20260811 10:16:48.110370 15376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:48.110702 15376 log.cc:824] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:48.111696 15376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: No bootstrap required, opened a new log
I20260811 10:16:48.114025 15376 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.114209 15376 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8ba0e2a364a43e7af501a5bd9fa507e, State: Initialized, Role: FOLLOWER
I20260811 10:16:48.114311 15376 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.114666 15535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.114743 15376 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:48.114769 15535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:16:48.116950 15557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:48.117017 15557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:48.117219 15263 master_runner.cc:186] Error getting master registration for 127.8.238.62:42377: OK, instance_id { permanent_uuid: "fa42b32b224646d9babf99a92186571e" instance_seqno: 1786443407673445 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:16:48.119871 15263 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:16:48.126355 15253 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } attempt: 3
Aug 11 10:16:48 dist-test-slave-7f01 krb5kdc[15116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443407, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:16:48.134660 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.126942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.8.238.60:36139)
0811 10:16:48.127003 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.127005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.127017 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:48.127052 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.127053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.127063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.127150 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.127540 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.128120 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.129150 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.129456 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.129521 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.131842 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.131844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.131844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.131859 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.133324 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.133788 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.133790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.133790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.133849 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.134056 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.134057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.134058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.134215 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.134280 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.134490 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.134572 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:16:48.134654 15257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.126824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:36139 (local address 127.0.0.1:44482)
0811 10:16:48.126905 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:48.126918 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:48.126968 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:48.127171 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:48.127172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:48.127395 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:48.127460 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.127462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.128209 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.128210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.129047 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:48.129049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.129521 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.129522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:48.129575 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.130098 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:48.130103 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:48.131709 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:48.133457 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.133459 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.133459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.133703 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.133924 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:48.133924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:48.133925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:48.133985 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:48.134273 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:48.134274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:48.134329 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 10:16:48.134407 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 10:16:48.134536 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 10:16:48.137106 15253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: Bootstrap starting.
I20260811 10:16:48.137965 15253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:48.138326 15253 log.cc:824] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:48.139153 15253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: No bootstrap required, opened a new log
I20260811 10:16:48.140724 15253 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.140890 15253 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa42b32b224646d9babf99a92186571e, State: Initialized, Role: FOLLOWER
I20260811 10:16:48.140985 15253 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.141374 15253 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:48.141433 15206 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 10:16:48.141479 15206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:48.142485 15206 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.142786 15206 leader_election.cc:288] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [CANDIDATE]: Term 1 election: Requested vote from peers d8ba0e2a364a43e7af501a5bd9fa507e (127.8.238.61:38955), 944b8a1b04204e6cbc0da4addc4a5e27 (127.8.238.60:36139)
I20260811 10:16:48.143311 15332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa42b32b224646d9babf99a92186571e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e"
I20260811 10:16:48.143479 15332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:48.141324 15562 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.144271 15562 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:48.144603 15332 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa42b32b224646d9babf99a92186571e in term 1.
I20260811 10:16:48.144928 15160 leader_election.cc:302] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8ba0e2a364a43e7af501a5bd9fa507e, fa42b32b224646d9babf99a92186571e; no voters:
I20260811 10:16:48.145206 15562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:48.145459 15562 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [term 1 LEADER]: Becoming Leader. State: Replica: fa42b32b224646d9babf99a92186571e, State: Running, Role: LEADER
I20260811 10:16:48.145402 15444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa42b32b224646d9babf99a92186571e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "944b8a1b04204e6cbc0da4addc4a5e27"
I20260811 10:16:48.145555 15444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:48.145689 15562 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } }
I20260811 10:16:48.146274 15562 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa42b32b224646d9babf99a92186571e. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.146428 15562 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:48.146682 15444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa42b32b224646d9babf99a92186571e in term 1.
W20260811 10:16:48.148214 15586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:48.148275 15586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:48.155532 15599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:48.155890 15599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:48.161834 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.148917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.8.238.60:36139)
0811 10:16:48.148959 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.148961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.153316 (+ 4355us) server_negotiation.cc:407] Connection header received
0811 10:16:48.153352 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.153353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.153366 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.153422 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.154224 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.155025 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.156330 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.156608 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.156660 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.158570 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.158573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.158574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.158585 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.160449 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.160896 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.160898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.160898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.160964 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.161175 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.161176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.161176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.161314 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.161395 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.161631 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.161725 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 10:16:48.165217 15444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [term 1 FOLLOWER]: Refusing update from remote peer fa42b32b224646d9babf99a92186571e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:48.165556 15332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [term 1 FOLLOWER]: Refusing update from remote peer fa42b32b224646d9babf99a92186571e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:48.166167 15562 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:48.166422 15562 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [LEADER]: Connected to new peer: Peer: permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:48.167917 15535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa42b32b224646d9babf99a92186571e. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.168025 15535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:48.168138 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa42b32b224646d9babf99a92186571e. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.168262 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:16:48.171082 15169 tablet.cc:2369] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:48.172606 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.149252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.8.238.62:42377)
0811 10:16:48.149290 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.149292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.153435 (+ 4143us) server_negotiation.cc:407] Connection header received
0811 10:16:48.159909 (+ 6474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.159911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.159924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.159957 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.162575 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.163491 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.166941 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.167225 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.167290 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.169205 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.169206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.169207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.169214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.170895 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.171638 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.171639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.171640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.171700 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.171944 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.171945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.171946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.172099 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.172189 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.172414 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.172506 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 10:16:48.172659 15386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:48.148709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.8.238.61:38955)
0811 10:16:48.148760 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.148762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.152087 (+ 3325us) server_negotiation.cc:407] Connection header received
0811 10:16:48.152187 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.152188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.152202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.152235 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.160711 (+ 8476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.161492 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.164745 (+ 3253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:48.165076 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:48.165121 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:48.168888 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:48.168890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:48.168890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:48.168899 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:48.170715 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.171649 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.171650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.171651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.171726 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:48.172039 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:48.172040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:48.172040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:48.172185 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:48.172266 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:48.172468 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 10:16:48.172560 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 10:16:48.174170 15562 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.174278 15562 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:48.174741 15599 catalog_manager.cc:1347] Generated new cluster ID: de48ff9308f54d3482e64f565741e23a
I20260811 10:16:48.174799 15599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:48.174726 15507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.174834 15507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:48.175030 15606 mvcc.cc:205] Tried to move back new op lower bound from 7317272199840800768 to 7317272199766028288. Current Snapshot: MvccSnapshot[applied={T|T < 7317272199840800768 or (T in {7317272199840800768})}]
I20260811 10:16:48.181159 15562 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.181310 15562 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:48.181908 15535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa42b32b224646d9babf99a92186571e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa42b32b224646d9babf99a92186571e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 42377 } } peers { permanent_uuid: "d8ba0e2a364a43e7af501a5bd9fa507e" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 38955 } } peers { permanent_uuid: "944b8a1b04204e6cbc0da4addc4a5e27" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 36139 } } }
I20260811 10:16:48.182026 15535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:48.182330 15607 mvcc.cc:205] Tried to move back new op lower bound from 7317272199840800768 to 7317272199766028288. Current Snapshot: MvccSnapshot[applied={T|T < 7317272199840800768}]
I20260811 10:16:48.183004 15609 mvcc.cc:205] Tried to move back new op lower bound from 7317272199840800768 to 7317272199766028288. Current Snapshot: MvccSnapshot[applied={T|T < 7317272199840800768}]
I20260811 10:16:48.208431 15599 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:48.209522 15599 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:48.220100 15599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa42b32b224646d9babf99a92186571e: Generated new TSK 0
I20260811 10:16:48.220376 15599 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:16:48.256421 15258 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.254482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.8.238.62:42377)
0811 10:16:48.254533 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.254535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.255543 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 10:16:48.255679 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.255681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.255693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.255724 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.256342 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58762: BlockingRecv error: recv got EOF from 127.0.0.1:58762 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 10:16:48.257288 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.254925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.8.238.61:38955)
0811 10:16:48.254983 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.254985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.256719 (+ 1734us) server_negotiation.cc:407] Connection header received
0811 10:16:48.256827 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.256829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.256844 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.256875 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.257222 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48376: BlockingRecv error: recv got EOF from 127.0.0.1:48376 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 10:16:48.258100 15499 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.255289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.8.238.60:36139)
0811 10:16:48.255342 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.255344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.257542 (+ 2198us) server_negotiation.cc:407] Connection header received
0811 10:16:48.257643 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.257645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.257658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.257690 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.258030 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44494: BlockingRecv error: recv got EOF from 127.0.0.1:44494 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 10:16:48.260993 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.260110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.8.238.61:38955)
0811 10:16:48.260154 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.260156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.260378 (+ 222us) server_negotiation.cc:407] Connection header received
0811 10:16:48.260497 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.260498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.260520 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.260555 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.260926 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48380: BlockingRecv error: recv got EOF from 127.0.0.1:48380 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 10:16:48.264395 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.263466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.8.238.61:38955)
0811 10:16:48.263506 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.263508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.263803 (+ 295us) server_negotiation.cc:407] Connection header received
0811 10:16:48.263916 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.263917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.263929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.263961 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.264339 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48390: BlockingRecv error: recv got EOF from 127.0.0.1:48390 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 10:16:48.268723 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.267710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.8.238.61:38955)
0811 10:16:48.267850 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.267852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.268134 (+ 282us) server_negotiation.cc:407] Connection header received
0811 10:16:48.268236 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.268237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.268251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.268285 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.268662 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48404: BlockingRecv error: recv got EOF from 127.0.0.1:48404 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 10:16:48.275223 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.274208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.8.238.61:38955)
0811 10:16:48.274258 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.274261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.274544 (+ 283us) server_negotiation.cc:407] Connection header received
0811 10:16:48.274663 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.274664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.274678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.274711 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.275155 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48408: BlockingRecv error: recv got EOF from 127.0.0.1:48408 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 10:16:48.286163 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.285061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48418 (local address 127.8.238.61:38955)
0811 10:16:48.285122 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.285124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.285433 (+ 309us) server_negotiation.cc:407] Connection header received
0811 10:16:48.285556 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.285558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.285572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.285607 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.286088 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48418: BlockingRecv error: recv got EOF from 127.0.0.1:48418 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 10:16:48.305413 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.304154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.8.238.61:38955)
0811 10:16:48.304207 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.304209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.304698 (+ 489us) server_negotiation.cc:407] Connection header received
0811 10:16:48.304839 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.304840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.304854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.304887 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.305336 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48432: BlockingRecv error: recv got EOF from 127.0.0.1:48432 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:16:48.340339 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.338997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.8.238.61:38955)
0811 10:16:48.339048 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.339051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.339307 (+ 256us) server_negotiation.cc:407] Connection header received
0811 10:16:48.339448 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.339449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.339467 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.339552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.340262 (+ 710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48444: BlockingRecv error: recv got EOF from 127.0.0.1:48444 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:16:48.406881 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.405847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48452 (local address 127.8.238.61:38955)
0811 10:16:48.405952 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.405955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.406052 (+ 97us) server_negotiation.cc:407] Connection header received
0811 10:16:48.406293 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.406294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.406309 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.406395 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.406789 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48452: BlockingRecv error: recv got EOF from 127.0.0.1:48452 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 10:16:48.537516 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.536568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.8.238.61:38955)
0811 10:16:48.536684 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.536687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.536804 (+ 117us) server_negotiation.cc:407] Connection header received
0811 10:16:48.536926 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.536927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.536940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.537018 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.537429 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48456: BlockingRecv error: recv got EOF from 127.0.0.1:48456 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 10:16:48.796710 15386 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:48.795585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.8.238.61:38955)
0811 10:16:48.795715 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:48.795718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:48.795733 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:16:48.795887 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:48.795889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:48.795911 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:48.795994 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:48.796567 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48460: BlockingRecv error: recv got EOF from 127.0.0.1:48460 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 10:16:49.105450 15530 catalog_manager.cc:1259] Loaded cluster ID: de48ff9308f54d3482e64f565741e23a
I20260811 10:16:49.105598 15530 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: loading cluster ID for follower catalog manager: success
I20260811 10:16:49.106511 15530 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: acquiring CA information for follower catalog manager: success
I20260811 10:16:49.106873 15530 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 944b8a1b04204e6cbc0da4addc4a5e27: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:16:49.117710 15557 catalog_manager.cc:1259] Loaded cluster ID: de48ff9308f54d3482e64f565741e23a
I20260811 10:16:49.117885 15557 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: loading cluster ID for follower catalog manager: success
I20260811 10:16:49.118820 15557 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: acquiring CA information for follower catalog manager: success
I20260811 10:16:49.119195 15557 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d8ba0e2a364a43e7af501a5bd9fa507e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:16:49.313055 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:49.310149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.8.238.61:38955)
0811 10:16:49.310279 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:49.310281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:49.310291 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:49.310323 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:49.310324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:49.310386 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:49.310490 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:49.310658 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.311195 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.312071 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.312283 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.312343 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:49.312514 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:49.312638 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:49.312710 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:49.312912 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 10:16:49.312935 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260811 10:16:49.318446 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:49.315491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.8.238.60:36139)
0811 10:16:49.315612 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:49.315613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:49.315622 (+ 9us) server_negotiation.cc:407] Connection header received
0811 10:16:49.315667 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:49.315668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:49.315710 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:49.315834 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:49.315956 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.316457 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.317277 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.317495 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.317556 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:49.317850 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:49.318041 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:49.318150 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:49.318248 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 10:16:49.318287 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260811 10:16:49.324337 15738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:49.321313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.8.238.60:36139)
0811 10:16:49.321403 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:49.321404 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:49.321411 (+ 7us) server_negotiation.cc:407] Connection header received
0811 10:16:49.321463 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:49.321464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:49.321477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:49.321535 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:49.321731 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.322487 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.323349 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.323641 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.323711 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:49.323907 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:49.323952 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:49.324006 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:49.324088 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 10:16:49.324114 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 10:16:49.324539 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:49.320915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.8.238.61:38955)
0811 10:16:49.320974 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:49.320976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:49.321136 (+ 160us) server_negotiation.cc:407] Connection header received
0811 10:16:49.321375 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:49.321377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:49.321392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:49.321454 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:49.321668 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.322487 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.323599 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.323956 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.324019 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:49.324207 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:49.324244 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:49.324274 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:49.324433 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 10:16:49.324465 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 10:16:49.324537 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:49.320960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.8.238.62:42377)
0811 10:16:49.321103 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:49.321105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:49.321230 (+ 125us) server_negotiation.cc:407] Connection header received
0811 10:16:49.321343 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:49.321344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:49.321410 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:49.321496 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:49.321693 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.322261 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.323399 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:49.323710 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:49.323813 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:49.324025 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:49.324153 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:49.324214 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:49.324273 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 10:16:49.324322 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":98,"threads_started":1}
I20260811 10:16:49.326619 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15139
I20260811 10:16:49.344906 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15263
I20260811 10:16:49.360739 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15389
2026-08-11T10:16:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3323 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 10:16:49 dist-test-slave-7f01 krb5kdc[15756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 10:16:49 dist-test-slave-7f01 krb5kdc[15756](info): set up 2 sockets
Aug 11 10:16:49 dist-test-slave-7f01 krb5kdc[15756](info): commencing operation
krb5kdc: starting...
W20260811 10:16:50.696717 9144 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/test-user.keytab.
Aug 11 10:16:50 dist-test-slave-7f01 krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T10:16:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T10:16:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.238.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
Entry for principal kudu/127.8.238.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab.
I20260811 10:16:50.848486 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39425
--webserver_interface=127.8.238.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 10:16:50.951313 15777 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:50.951658 15777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:50.951694 15777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:50.951772 15777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:50.953173 15777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:50.953266 15777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:50.953294 15777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:50.953327 15777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:50.953356 15777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:50.955127 15777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.62.keytab
--principal=kudu/127.8.238.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.62:39425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.238.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:50.955487 15777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:50.955822 15777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:50.961571 15791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:50.961571 15786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:50.961777 15787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:50.961900 15777 server_base.cc:1034] running on GCE node
I20260811 10:16:50.962251 15777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:50.962658 15777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:50.963871 15777 hybrid_clock.cc:648] HybridClock initialized: now 1786443410963799 us; error 82 us; skew 500 ppm
Aug 11 10:16:50 dist-test-slave-7f01 krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:50.967468 15777 init.cc:377] Logged in from keytab as kudu/127.8.238.62@KRBTEST.COM (short username kudu)
I20260811 10:16:50.968137 15777 webserver.cc:460] Webserver started at http://127.8.238.62:33287/ using document root <none> and password file <none>
I20260811 10:16:50.968410 15777 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:50.968487 15777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:50.968654 15777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:50.969909 15777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data/instance:
uuid: "67e07aac728449b38fa9b713b37bcf7e"
format_stamp: "Formatted at 2026-08-11 10:16:50 on dist-test-slave-7f01"
server_key: "14ff79b1d833ef02ada515afa419e20b"
server_key_iv: "93f3f09c2f38ec957b72e5990b5f3030"
server_key_version: "encryptionkey@0"
I20260811 10:16:50.970345 15777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance:
uuid: "67e07aac728449b38fa9b713b37bcf7e"
format_stamp: "Formatted at 2026-08-11 10:16:50 on dist-test-slave-7f01"
server_key: "14ff79b1d833ef02ada515afa419e20b"
server_key_iv: "93f3f09c2f38ec957b72e5990b5f3030"
server_key_version: "encryptionkey@0"
I20260811 10:16:50.972128 15777 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 10:16:50.973265 15801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:50.973513 15777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:50.973645 15777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal
uuid: "67e07aac728449b38fa9b713b37bcf7e"
format_stamp: "Formatted at 2026-08-11 10:16:50 on dist-test-slave-7f01"
server_key: "14ff79b1d833ef02ada515afa419e20b"
server_key_iv: "93f3f09c2f38ec957b72e5990b5f3030"
server_key_version: "encryptionkey@0"
I20260811 10:16:50.973737 15777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:51.006353 15777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:51.009339 15777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:51.009579 15777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:51.015959 15777 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.62:39425
I20260811 10:16:51.015985 15895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.62:39425 every 8 connection(s)
I20260811 10:16:51.016450 15777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/data/info.pb
I20260811 10:16:51.017585 15896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:51.017671 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15777
I20260811 10:16:51.017824 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-0/wal/instance
I20260811 10:16:51.018075 9144 external_mini_cluster.cc:1428] Setting key 3ed5539bf219c528878f3f858e33c821
I20260811 10:16:51.019547 15896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.61@KRBTEST.COM" created.
I20260811 10:16:51.032852 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.020613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.8.238.62:39425)
0811 10:16:51.020810 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.020823 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.020984 (+ 161us) server_negotiation.cc:407] Connection header received
0811 10:16:51.021623 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.021626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.021651 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.021772 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.022481 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.023993 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.025096 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.025675 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.025805 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.028649 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.028661 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.028661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.028671 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.030629 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.031139 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.031143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.031143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.031293 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.031687 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.031688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.031689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.032071 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.032178 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.032499 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.032658 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260811 10:16:51.033119 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.020571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39425 (local address 127.0.0.1:41918)
0811 10:16:51.020830 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.020881 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.021443 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.021823 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.021826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.022170 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.022349 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.022354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.024147 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.024149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.024992 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.024995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.025871 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.025872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.025937 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.026651 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.026661 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.028542 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.030748 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.030757 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.030760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.031009 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.031488 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.031489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.031489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.031607 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.032177 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.032186 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.032315 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.032461 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.032768 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":83,"thread_start_us":114,"threads_started":1}
I20260811 10:16:51.035439 15896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:51.036142 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.035934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45178)
0811 10:16:51.036017 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.036097 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 10:16:51.036410 15806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
I20260811 10:16:51.037146 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.036988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45190)
0811 10:16:51.037060 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.037104 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 10:16:51.037395 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33359: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
Entry for principal kudu/127.8.238.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab.
I20260811 10:16:51.051510 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:33359
--webserver_interface=127.8.238.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:16:51.086764 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } attempt: 1
I20260811 10:16:51.087466 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.087293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45198)
0811 10:16:51.087372 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.087424 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 10:16:51.088042 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.087928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45208)
0811 10:16:51.087983 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.088016 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:16:51.088196 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33359: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
W20260811 10:16:51.146333 15907 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:51.146607 15907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:51.146626 15907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:51.146649 15907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:51.147912 15907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:51.147965 15907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:51.147979 15907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:51.147992 15907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:51.148005 15907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:51.149663 15907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.61.keytab
--principal=kudu/127.8.238.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.61:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.238.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:51.149905 15907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:51.150104 15907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 10:16:51.155786 15915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:51.155992 15907 server_base.cc:1034] running on GCE node
W20260811 10:16:51.155788 15921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:51.155884 15917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:51.156328 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } attempt: 2
I20260811 10:16:51.156780 15907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:51.157006 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.156785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45224)
0811 10:16:51.156906 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.156957 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 10:16:51.157270 15907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:51.157583 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.157357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45232)
0811 10:16:51.157539 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.157562 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260811 10:16:51.157691 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.61:33359: Network error: Client connection negotiation failed: client connection to 127.8.238.61:33359: connect: Connection refused (error 111)
I20260811 10:16:51.158452 15907 hybrid_clock.cc:648] HybridClock initialized: now 1786443411158418 us; error 46 us; skew 500 ppm
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:51.161429 15907 init.cc:377] Logged in from keytab as kudu/127.8.238.61@KRBTEST.COM (short username kudu)
I20260811 10:16:51.161998 15907 webserver.cc:460] Webserver started at http://127.8.238.61:43663/ using document root <none> and password file <none>
I20260811 10:16:51.162273 15907 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:51.162349 15907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:51.162519 15907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:51.163720 15907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data/instance:
uuid: "34382fb050d946e68581751161ae3ecd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "6e335bfc949f7498dc16b5063e1c47ae"
server_key_iv: "536f955fc963b8b598669177e7159197"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.164213 15907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal/instance:
uuid: "34382fb050d946e68581751161ae3ecd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "6e335bfc949f7498dc16b5063e1c47ae"
server_key_iv: "536f955fc963b8b598669177e7159197"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.165766 15907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 10:16:51.166702 15938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:51.166972 15907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 10:16:51.167217 15907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal
uuid: "34382fb050d946e68581751161ae3ecd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "6e335bfc949f7498dc16b5063e1c47ae"
server_key_iv: "536f955fc963b8b598669177e7159197"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.167418 15907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:51.177933 15907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:51.180601 15907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:51.180797 15907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:51.186012 15907 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.61:33359
I20260811 10:16:51.186075 16021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.61:33359 every 8 connection(s)
I20260811 10:16:51.186511 15907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/data/info.pb
I20260811 10:16:51.187698 16022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:51.189702 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 15907
I20260811 10:16:51.189850 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-1/wal/instance
I20260811 10:16:51.190114 9144 external_mini_cluster.cc:1428] Setting key 441971d6beb55eb2f63c9f2c14366d84
I20260811 10:16:51.190507 16022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.238.60@KRBTEST.COM; defaulting to no policy
I20260811 10:16:51.205279 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.191815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.8.238.62:39425)
0811 10:16:51.191901 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.191904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.192190 (+ 286us) server_negotiation.cc:407] Connection header received
0811 10:16:51.193355 (+ 1165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.193356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.193373 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.193433 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.194387 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.195282 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.196826 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.197269 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.197394 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.200955 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.200959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.200960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.200976 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.203146 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.203916 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.203918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.203919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.204010 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.204331 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.204332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.204333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.204493 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.204584 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.205048 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.205165 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 10:16:51.205470 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.191697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39425 (local address 127.0.0.1:41930)
0811 10:16:51.191989 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.192053 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.193200 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.193550 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.193555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.193985 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.194256 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.194261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.195505 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.195507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.196668 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.196673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.197466 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.197468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.197630 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.198530 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.198548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.200774 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.203355 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.203375 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.203379 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.203801 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.204092 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.204094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.204094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.204243 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.204575 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.204584 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.204818 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.204954 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.205179 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":143,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.238.60@KRBTEST.COM" created.
I20260811 10:16:51.206188 16022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:51.215382 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.206575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45238)
0811 10:16:51.206638 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.206651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.206711 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.207185 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.207186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.207343 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.207416 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.207418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.208889 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.208890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.209475 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.209477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.210057 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.210058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.210136 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.210700 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.210705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.212108 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.214171 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.214172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.214173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.214355 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.214601 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.214601 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.214602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.214663 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.214987 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.214988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.215036 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.215126 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.215252 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 10:16:51.215405 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.206671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.8.238.61:33359)
0811 10:16:51.206976 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.206982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.207013 (+ 31us) server_negotiation.cc:407] Connection header received
0811 10:16:51.207041 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.207044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.207062 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.207156 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.207513 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.208778 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.209586 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.209962 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.210061 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.212223 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.212243 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.212243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.212249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.214047 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.214463 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.214467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.214467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.214529 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.214728 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.214729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.214730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.214905 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.215001 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.215162 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.215306 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":263,"thread_start_us":201,"threads_started":1}
I20260811 10:16:51.216281 16022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:51.217098 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.216884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48860)
0811 10:16:51.216996 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.217053 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 10:16:51.217443 15939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
I20260811 10:16:51.218371 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.218203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48862)
0811 10:16:51.218290 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.218339 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 10:16:51.218593 16022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
Entry for principal kudu/127.8.238.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab.
I20260811 10:16:51.227905 9144 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6YJKWE/build/release/bin/kudu
/tmp/dist-test-task6YJKWE/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:43121
--webserver_interface=127.8.238.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.238.20:34379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 10:16:51.248863 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } attempt: 3
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:51.258097 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.249419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.8.238.61:33359)
0811 10:16:51.249499 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.249501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.249516 (+ 15us) server_negotiation.cc:407] Connection header received
0811 10:16:51.249552 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.249553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.249573 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.249642 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.250003 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.250657 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.251458 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.251690 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.251862 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.254026 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.254028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.254029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.254037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.256306 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.256846 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.256847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.256847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.256911 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.257288 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.257291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.257291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.257550 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.257679 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.257926 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.258021 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 10:16:51.258111 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.249337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45254)
0811 10:16:51.249407 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.249419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.249488 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.249647 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.249648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.249839 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.249914 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.249916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.250789 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.250791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.251352 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.251354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.251863 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.251864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.251959 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.252475 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.252481 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.253897 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.256436 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.256437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.256438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.256752 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.256988 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.256989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.256990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.257068 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.257687 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.257688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.257763 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.257862 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.257996 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 10:16:51.258667 15896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:51.259089 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.258954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48870)
0811 10:16:51.259013 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.259058 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:16:51.259464 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.259363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48880)
0811 10:16:51.259415 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.259446 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 10:16:51.259616 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
I20260811 10:16:51.267889 16022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 1
I20260811 10:16:51.268579 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.268400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48894)
0811 10:16:51.268480 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.268535 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 10:16:51.269006 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.268892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48900)
0811 10:16:51.268949 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.268982 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 10:16:51.269232 16022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
I20260811 10:16:51.290800 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 1
I20260811 10:16:51.291458 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.291277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48904)
0811 10:16:51.291356 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.291413 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 10:16:51.291919 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.291828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48912)
0811 10:16:51.291862 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.291896 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 10:16:51.292026 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
W20260811 10:16:51.329238 16035 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 10:16:51.329514 16035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 10:16:51.329532 16035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 10:16:51.329556 16035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 10:16:51.330935 16035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 10:16:51.330999 16035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 10:16:51.331015 16035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 10:16:51.331028 16035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 10:16:51.331043 16035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 10:16:51.332772 16035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.238.20:34379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.238.62:39425,127.8.238.61:33359,127.8.238.60:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/krb5kdc/kudu_127.8.238.60.keytab
--principal=kudu/127.8.238.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.238.60:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.238.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 08:54:39 UTC on ccycloud.quasar-rkjucd.root.comops.site
build id 4579928
I20260811 10:16:51.333099 16035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 10:16:51.333388 16035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 10:16:51.337378 16022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 2
I20260811 10:16:51.338063 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.337813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48916)
0811 10:16:51.337913 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.337975 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 10:16:51.338611 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.338432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48928)
0811 10:16:51.338524 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.338582 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 10:16:51.338907 16052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:51.339052 16022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
W20260811 10:16:51.338894 16050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 10:16:51.338958 16044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 10:16:51.339669 16035 server_base.cc:1034] running on GCE node
I20260811 10:16:51.340111 16035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 10:16:51.340471 16035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 10:16:51.341668 16035 hybrid_clock.cc:648] HybridClock initialized: now 1786443411341652 us; error 64 us; skew 500 ppm
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 10:16:51.345522 16035 init.cc:377] Logged in from keytab as kudu/127.8.238.60@KRBTEST.COM (short username kudu)
I20260811 10:16:51.346184 16035 webserver.cc:460] Webserver started at http://127.8.238.60:35181/ using document root <none> and password file <none>
I20260811 10:16:51.346487 16035 fs_manager.cc:359] Metadata directory not provided
I20260811 10:16:51.346566 16035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 10:16:51.346837 16035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 10:16:51.348569 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data/instance:
uuid: "59d6458180ce45c2a5f01960e4de32bd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "131a9902a101dfc88d9dd8699cded317"
server_key_iv: "fca1fa0fa5c4a707cabaf7b88ccd9542"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.349093 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal/instance:
uuid: "59d6458180ce45c2a5f01960e4de32bd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "131a9902a101dfc88d9dd8699cded317"
server_key_iv: "fca1fa0fa5c4a707cabaf7b88ccd9542"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.350746 16035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 10:16:51.351868 16065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 10:16:51.352079 16035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 10:16:51.352198 16035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data,/tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal
uuid: "59d6458180ce45c2a5f01960e4de32bd"
format_stamp: "Formatted at 2026-08-11 10:16:51 on dist-test-slave-7f01"
server_key: "131a9902a101dfc88d9dd8699cded317"
server_key_iv: "fca1fa0fa5c4a707cabaf7b88ccd9542"
server_key_version: "encryptionkey@0"
I20260811 10:16:51.352294 16035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 10:16:51.367211 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 2
I20260811 10:16:51.367928 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.367668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48930)
0811 10:16:51.367762 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.367890 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 10:16:51.368459 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.368341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48946)
0811 10:16:51.368407 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.368437 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 10:16:51.368588 15896 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.238.60:43121: Network error: Client connection negotiation failed: client connection to 127.8.238.60:43121: connect: Connection refused (error 111)
I20260811 10:16:51.370113 16035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 10:16:51.372956 16035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 10:16:51.373086 16035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 10:16:51.377830 16035 rpc_server.cc:307] RPC server started. Bound to: 127.8.238.60:43121
I20260811 10:16:51.377931 16147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.238.60:43121 every 8 connection(s)
I20260811 10:16:51.378360 16035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/data/info.pb
I20260811 10:16:51.379541 16149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 10:16:51.381371 16149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } has no permanent_uuid. Determining permanent_uuid...
I20260811 10:16:51.386327 9144 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6YJKWE/build/release/bin/kudu as pid 16035
I20260811 10:16:51.386471 9144 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6YJKWE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786443124069174-9144-0/minicluster-data/master-2/wal/instance
I20260811 10:16:51.386772 9144 external_mini_cluster.cc:1428] Setting key 3930b3288b2bf5e2a7b7f243b6f4f93d
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
I20260811 10:16:51.392874 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.382304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.8.238.62:39425)
0811 10:16:51.382406 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.382409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.382615 (+ 206us) server_negotiation.cc:407] Connection header received
0811 10:16:51.383204 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.383206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.383219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.383271 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.384059 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.384580 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.385623 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.385828 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.385921 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.389256 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.389258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.389258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.389266 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.390943 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.391508 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.391510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.391510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.391577 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.391930 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.391932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.391932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.392091 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.392169 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.392527 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.392653 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 10:16:51.392871 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.382203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.62:39425 (local address 127.0.0.1:41934)
0811 10:16:51.382481 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.382528 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.383067 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.383353 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.383357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.383656 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.383955 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.383960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.384693 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.384695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.385527 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.385529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.385950 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.385952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.386121 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.386829 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.386840 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.389068 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.391041 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.391049 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.391052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.391399 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.391690 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.391691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.391691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.391864 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.392171 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.392178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.392358 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.392480 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.392674 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":124,"threads_started":1}
I20260811 10:16:51.393611 16149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:51.400069 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.387331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.8.238.60:43121)
0811 10:16:51.387518 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.387523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.387541 (+ 18us) server_negotiation.cc:407] Connection header received
0811 10:16:51.388295 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.388302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.388345 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.388417 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.389516 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.390574 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.391651 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.392219 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.392287 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.396113 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.396124 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.396124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.396132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.398423 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.398954 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.398966 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.398967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.399045 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.399281 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.399283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.399283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.399489 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.399694 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.399781 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.399890 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260811 10:16:51.403019 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.394043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45256 (local address 127.8.238.61:33359)
0811 10:16:51.394087 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.394089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.394097 (+ 8us) server_negotiation.cc:407] Connection header received
0811 10:16:51.394133 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.394134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.394142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.394191 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.394510 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.395235 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.396540 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.396858 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.396954 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.399320 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.399328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.399328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.399338 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.401359 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.401962 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.401965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.401966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.402070 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.402316 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.402317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.402317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.402482 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.402588 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.402772 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.402914 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 10:16:51.403026 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.393973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.61:33359 (local address 127.0.0.1:45256)
0811 10:16:51.394027 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.394037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.394074 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.394200 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.394201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.394371 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.394435 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.394437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.395394 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.395397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.396421 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.396428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.396987 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.396989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.397115 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.397699 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.397705 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.399193 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.401506 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.401509 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.401509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.401816 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.402159 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.402160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.402160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.402235 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.402574 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.402575 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.402648 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.402739 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.402892 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 10:16:51.403549 16149 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.62@KRBTEST.COM
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.60@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:16:51.410804 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.401373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.8.238.62:39425)
0811 10:16:51.401488 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.401491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.401502 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:51.401536 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.401538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.401555 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.401629 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.402188 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.402768 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.403871 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.404197 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.404291 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.407472 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.407475 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.407475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.407492 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.409164 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.409617 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.409620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.409620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.409755 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.410132 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.410135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.410135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.410350 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.410519 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.410653 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.410729 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 10:16:51.412586 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.404055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.8.238.60:43121)
0811 10:16:51.404139 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.404142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.404172 (+ 30us) server_negotiation.cc:407] Connection header received
0811 10:16:51.404193 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.404194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.404208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.404294 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.404639 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.405533 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.406404 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.406671 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.406777 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.409054 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.409056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.409056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.409064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.411321 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.411675 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.411676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.411676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.411734 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.411976 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.411977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.411977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.412107 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.412186 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.412339 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.412464 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 10:16:51.412657 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.403976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48954)
0811 10:16:51.404027 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.404038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.404114 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.404300 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.404301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.404482 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.404529 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.404531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.405666 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.405668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.406266 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.406267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.406772 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.406774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.406828 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.407384 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.407397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.408955 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.411408 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.411408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.411408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.411590 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.411842 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.411843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.411843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.411910 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.412177 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.412178 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.412222 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.412315 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.412527 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":103}
I20260811 10:16:51.415261 16149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: Bootstrap starting.
I20260811 10:16:51.416349 16149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:51.416704 16149 log.cc:824] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: Log is configured to *not* fsync() on all Append() calls
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.238.61@KRBTEST.COM
I20260811 10:16:51.417730 16149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: No bootstrap required, opened a new log
I20260811 10:16:51.419456 16149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.419664 16149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59d6458180ce45c2a5f01960e4de32bd, State: Initialized, Role: FOLLOWER
I20260811 10:16:51.419840 16149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.420264 16149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:51.420243 16156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.420439 16156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:51.420904 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.411676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.8.238.61:33359)
0811 10:16:51.411726 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.411728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.411802 (+ 74us) server_negotiation.cc:407] Connection header received
0811 10:16:51.411883 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.411884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.411894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.411950 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.412481 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.413215 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.413964 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.414232 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.414313 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.417511 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.417513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.417513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.417530 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.419230 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.419775 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.419776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.419777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.419852 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.420129 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.420130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.420130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.420310 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.420650 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.420689 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.420767 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:16:51.422369 16174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:51.422451 16174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:51.423452 9144 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 10:16:51.424576 15846 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "67e07aac728449b38fa9b713b37bcf7e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41940
I20260811 10:16:51.423069 16035 master_runner.cc:186] Error getting master registration for 127.8.238.62:39425: OK, instance_id { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" instance_seqno: 1786443411014434 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:16:51.426577 16035 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:16:51.430284 16022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 3
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.61@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:16:51.440478 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.430738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48968)
0811 10:16:51.430794 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.430809 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.430897 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.431098 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.431099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.431309 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.431363 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.431365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.432458 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.432460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.433029 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.433031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.433570 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.433571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.433642 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.434225 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.434231 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.436062 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.438868 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.438870 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.438870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.439180 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.439467 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.439469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.439469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.439552 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.440056 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.440057 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.440129 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.440220 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.440354 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 10:16:51.440654 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.430849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.8.238.60:43121)
0811 10:16:51.430935 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.430938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.430950 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:51.430982 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.430984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.431003 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.431075 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.431475 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.432377 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.433139 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.433478 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.433549 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.436231 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.436235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.436235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.436250 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.438709 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.439293 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.439294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.439295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.439377 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.439640 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.439641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.439641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.439965 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.440054 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.440269 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.440389 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 10:16:51.443552 16022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: Bootstrap starting.
I20260811 10:16:51.444494 16022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:51.444880 16022 log.cc:824] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:51.445886 16022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: No bootstrap required, opened a new log
I20260811 10:16:51.447997 16022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.448212 16022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34382fb050d946e68581751161ae3ecd, State: Initialized, Role: FOLLOWER
I20260811 10:16:51.448324 16022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.448690 16179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.448784 16022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:51.448796 16179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:16:51.450793 16197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:51.450874 16197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:51.451149 15907 master_runner.cc:186] Error getting master registration for 127.8.238.62:39425: OK, instance_id { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" instance_seqno: 1786443411014434 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 10:16:51.454403 15907 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:16:51.467784 15896 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } attempt: 3
Aug 11 10:16:51 dist-test-slave-7f01 krb5kdc[15756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786443410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.238.62@KRBTEST.COM for kudu/127.8.238.60@KRBTEST.COM
I20260811 10:16:51.476583 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.468244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.238.60:43121 (local address 127.0.0.1:48974)
0811 10:16:51.468314 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 10:16:51.468327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 10:16:51.468390 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 10:16:51.468634 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 10:16:51.468635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 10:16:51.468831 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 10:16:51.468894 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.468897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.469859 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.469861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.470507 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 10:16:51.470510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.471016 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.471018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 10:16:51.471074 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.471664 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 10:16:51.471670 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 10:16:51.473291 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 10:16:51.475381 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.475382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.475383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.475583 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.475835 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 10:16:51.475836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 10:16:51.475836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 10:16:51.475900 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 10:16:51.476199 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 10:16:51.476201 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 10:16:51.476249 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 10:16:51.476334 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 10:16:51.476482 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 10:16:51.476584 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.468343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.8.238.60:43121)
0811 10:16:51.468502 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.468505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.468515 (+ 10us) server_negotiation.cc:407] Connection header received
0811 10:16:51.468547 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.468548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.468559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.468615 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.468991 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.469763 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.470622 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.470947 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.471008 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.473418 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.473420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.473420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.473429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.475289 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.475680 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.475681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.475681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.475769 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.475970 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.475971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.475972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.476133 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.476196 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.476376 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.476472 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 10:16:51.478933 15896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: Bootstrap starting.
I20260811 10:16:51.479964 15896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: Neither blocks nor log segments found. Creating new log.
I20260811 10:16:51.480324 15896 log.cc:824] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: Log is configured to *not* fsync() on all Append() calls
I20260811 10:16:51.481217 15896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: No bootstrap required, opened a new log
I20260811 10:16:51.482740 15896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.482908 15896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e07aac728449b38fa9b713b37bcf7e, State: Initialized, Role: FOLLOWER
I20260811 10:16:51.483004 15896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.483402 15896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 10:16:51.483457 15846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 10:16:51.483505 15846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:51.484612 15846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.484938 15846 leader_election.cc:288] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [CANDIDATE]: Term 1 election: Requested vote from peers 34382fb050d946e68581751161ae3ecd (127.8.238.61:33359), 59d6458180ce45c2a5f01960e4de32bd (127.8.238.60:43121)
W20260811 10:16:51.485590 16220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 10:16:51.485652 16220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 10:16:51.485904 15981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67e07aac728449b38fa9b713b37bcf7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "34382fb050d946e68581751161ae3ecd"
I20260811 10:16:51.486094 15981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:51.487259 15981 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67e07aac728449b38fa9b713b37bcf7e in term 1.
I20260811 10:16:51.487633 16110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67e07aac728449b38fa9b713b37bcf7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "59d6458180ce45c2a5f01960e4de32bd"
I20260811 10:16:51.487807 16110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 10:16:51.488904 16110 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67e07aac728449b38fa9b713b37bcf7e in term 1.
I20260811 10:16:51.489580 15803 leader_election.cc:302] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59d6458180ce45c2a5f01960e4de32bd, 67e07aac728449b38fa9b713b37bcf7e; no voters:
I20260811 10:16:51.483386 16202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.490499 16202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:51.490621 16202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 10:16:51.490720 16202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [term 1 LEADER]: Becoming Leader. State: Replica: 67e07aac728449b38fa9b713b37bcf7e, State: Running, Role: LEADER
I20260811 10:16:51.490871 16202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } }
I20260811 10:16:51.491488 16202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67e07aac728449b38fa9b713b37bcf7e. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.491639 16202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:51.492033 15777 master_runner.cc:263] No leader master found from master 67e07aac728449b38fa9b713b37bcf7e
I20260811 10:16:51.492089 15777 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 10:16:51.495042 16233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 10:16:51.495285 16233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 10:16:51.499030 16110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [term 1 FOLLOWER]: Refusing update from remote peer 67e07aac728449b38fa9b713b37bcf7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:51.499392 16202 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:51.500849 16156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67e07aac728449b38fa9b713b37bcf7e. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.500993 16156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:51.501480 15981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [term 1 FOLLOWER]: Refusing update from remote peer 67e07aac728449b38fa9b713b37bcf7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 10:16:51.501778 16202 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 10:16:51.503185 16179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67e07aac728449b38fa9b713b37bcf7e. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.503293 16179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: This master's current role is: FOLLOWER
W20260811 10:16:51.510797 15809 tablet.cc:2369] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 10:16:51.512256 16202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.512405 16202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:51.513145 16156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.513273 16156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:51.513612 16234 mvcc.cc:205] Tried to move back new op lower bound from 7317272213497020416 to 7317272213468033024. Current Snapshot: MvccSnapshot[applied={T|T < 7317272213497020416}]
I20260811 10:16:51.513808 16240 mvcc.cc:205] Tried to move back new op lower bound from 7317272213497020416 to 7317272213468033024. Current Snapshot: MvccSnapshot[applied={T|T < 7317272213497020416}]
I20260811 10:16:51.520648 16233 catalog_manager.cc:1347] Generated new cluster ID: 3eaee30c24754470a48cfde6bf4404c1
I20260811 10:16:51.520730 16233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 10:16:51.529294 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.497645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.8.238.62:39425)
0811 10:16:51.497692 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.497694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.504730 (+ 7036us) server_negotiation.cc:407] Connection header received
0811 10:16:51.504800 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.504801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.504815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.504872 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.511155 (+ 6283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.511924 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.512960 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.513278 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.513321 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.514821 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.514823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.514823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.514830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.518201 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.519972 (+ 1771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.519975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.519975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.520046 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.520311 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.520312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.520312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.520462 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.520549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.529035 (+ 8486us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.529176 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 10:16:51.530421 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.497796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.8.238.60:43121)
0811 10:16:51.497852 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.497854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.506876 (+ 9022us) server_negotiation.cc:407] Connection header received
0811 10:16:51.506986 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.506987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.507000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.507032 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.508859 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.509603 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.512064 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.512350 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.512400 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.515186 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.515187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.515188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.515195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.517255 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.528168 (+ 10913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.528171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.528171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.528297 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.528522 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.528522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.528523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.528670 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.528783 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.530208 (+ 1425us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.530313 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:16:51.532011 16202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.532131 16202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e [sys.catalog]: This master's current role is: LEADER
I20260811 10:16:51.532815 16244 mvcc.cc:205] Tried to move back new op lower bound from 7317272213497020416 to 7317272213468033024. Current Snapshot: MvccSnapshot[applied={T|T < 7317272213497020416}]
I20260811 10:16:51.533041 16179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67e07aac728449b38fa9b713b37bcf7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67e07aac728449b38fa9b713b37bcf7e" member_type: VOTER last_known_addr { host: "127.8.238.62" port: 39425 } } peers { permanent_uuid: "34382fb050d946e68581751161ae3ecd" member_type: VOTER last_known_addr { host: "127.8.238.61" port: 33359 } } peers { permanent_uuid: "59d6458180ce45c2a5f01960e4de32bd" member_type: VOTER last_known_addr { host: "127.8.238.60" port: 43121 } } }
I20260811 10:16:51.533138 16179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 10:16:51.534461 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:51.497133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45274 (local address 127.8.238.61:33359)
0811 10:16:51.497187 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.497189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.503543 (+ 6354us) server_negotiation.cc:407] Connection header received
0811 10:16:51.503654 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.503655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.503667 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.503698 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.504801 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.505632 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.507861 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:51.508189 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:51.508235 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:51.514164 (+ 5929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 10:16:51.514166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 10:16:51.514167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 10:16:51.514177 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 10:16:51.519142 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.533454 (+ 14312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.533456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.533456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.533550 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 10:16:51.533772 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 10:16:51.533773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 10:16:51.533773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 10:16:51.533940 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 10:16:51.534052 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:51.534262 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 10:16:51.534363 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 10:16:51.564320 16233 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 10:16:51.565240 16233 catalog_manager.cc:1504] Loading token signing keys...
I20260811 10:16:51.572374 16233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67e07aac728449b38fa9b713b37bcf7e: Generated new TSK 0
I20260811 10:16:51.572842 16233 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 10:16:51.605679 15902 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.603436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.8.238.62:39425)
0811 10:16:51.603488 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.603490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.604893 (+ 1403us) server_negotiation.cc:407] Connection header received
0811 10:16:51.605010 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.605011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.605024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.605056 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.605591 (+ 535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41966: BlockingRecv error: recv got EOF from 127.0.0.1:41966 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:16:51.606745 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.603803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.8.238.61:33359)
0811 10:16:51.603854 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.603856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.606113 (+ 2257us) server_negotiation.cc:407] Connection header received
0811 10:16:51.606222 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.606224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.606237 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.606268 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.606674 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45288: BlockingRecv error: recv got EOF from 127.0.0.1:45288 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:16:51.607724 16153 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.604489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.8.238.60:43121)
0811 10:16:51.604537 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.604539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.607162 (+ 2623us) server_negotiation.cc:407] Connection header received
0811 10:16:51.607260 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.607261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.607274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.607304 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.607656 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48994: BlockingRecv error: recv got EOF from 127.0.0.1:48994 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 10:16:51.610707 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.609813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.8.238.61:33359)
0811 10:16:51.609857 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.609859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.610078 (+ 219us) server_negotiation.cc:407] Connection header received
0811 10:16:51.610203 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.610205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.610226 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.610258 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.610651 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45300: BlockingRecv error: recv got EOF from 127.0.0.1:45300 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 10:16:51.614280 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.613364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.8.238.61:33359)
0811 10:16:51.613410 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.613412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.613674 (+ 262us) server_negotiation.cc:407] Connection header received
0811 10:16:51.613782 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.613783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.613796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.613828 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.614218 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45302: BlockingRecv error: recv got EOF from 127.0.0.1:45302 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:16:51.618832 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.617831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45304 (local address 127.8.238.61:33359)
0811 10:16:51.617876 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.617878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.618172 (+ 294us) server_negotiation.cc:407] Connection header received
0811 10:16:51.618284 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.618286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.618298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.618331 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.618769 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45304: BlockingRecv error: recv got EOF from 127.0.0.1:45304 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 10:16:51.625653 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.624561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.8.238.61:33359)
0811 10:16:51.624614 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.624617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.624936 (+ 319us) server_negotiation.cc:407] Connection header received
0811 10:16:51.625053 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.625055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.625067 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.625099 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.625584 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45316: BlockingRecv error: recv got EOF from 127.0.0.1:45316 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 10:16:51.636848 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.635332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.8.238.61:33359)
0811 10:16:51.635399 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.635401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.635967 (+ 566us) server_negotiation.cc:407] Connection header received
0811 10:16:51.636061 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.636064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.636089 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.636212 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.636758 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45328: BlockingRecv error: recv got EOF from 127.0.0.1:45328 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 10:16:51.655051 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.654140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45330 (local address 127.8.238.61:33359)
0811 10:16:51.654194 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.654197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.654366 (+ 169us) server_negotiation.cc:407] Connection header received
0811 10:16:51.654436 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.654437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.654452 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.654525 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.654960 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45330: BlockingRecv error: recv got EOF from 127.0.0.1:45330 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 10:16:51.689468 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.688400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.8.238.61:33359)
0811 10:16:51.688567 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.688570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.688582 (+ 12us) server_negotiation.cc:407] Connection header received
0811 10:16:51.688623 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.688624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.688634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.688704 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.689364 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45334: BlockingRecv error: recv got EOF from 127.0.0.1:45334 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
W20260811 10:16:51.755928 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.754889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.8.238.61:33359)
0811 10:16:51.754962 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.754964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.755116 (+ 152us) server_negotiation.cc:407] Connection header received
0811 10:16:51.755154 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.755155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.755169 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.755228 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.755821 (+ 593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45348: BlockingRecv error: recv got EOF from 127.0.0.1:45348 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260811 10:16:51.887136 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:51.885557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.8.238.61:33359)
0811 10:16:51.885676 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:51.885679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:51.885960 (+ 281us) server_negotiation.cc:407] Connection header received
0811 10:16:51.886172 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:51.886175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:51.886194 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:51.886263 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:51.887035 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45350: BlockingRecv error: recv got EOF from 127.0.0.1:45350 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 10:16:52.146118 16027 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 10:16:52.144930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.8.238.61:33359)
0811 10:16:52.145002 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.145004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.145081 (+ 77us) server_negotiation.cc:407] Connection header received
0811 10:16:52.145167 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.145168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.145180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.145240 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 10:16:52.145971 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45354: BlockingRecv error: recv got EOF from 127.0.0.1:45354 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 10:16:52.423120 16174 catalog_manager.cc:1259] Loaded cluster ID: 3eaee30c24754470a48cfde6bf4404c1
I20260811 10:16:52.423244 16174 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: loading cluster ID for follower catalog manager: success
I20260811 10:16:52.424769 16174 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: acquiring CA information for follower catalog manager: success
I20260811 10:16:52.425230 16174 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59d6458180ce45c2a5f01960e4de32bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:16:52.451555 16197 catalog_manager.cc:1259] Loaded cluster ID: 3eaee30c24754470a48cfde6bf4404c1
I20260811 10:16:52.451829 16197 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: loading cluster ID for follower catalog manager: success
I20260811 10:16:52.453083 16197 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: acquiring CA information for follower catalog manager: success
I20260811 10:16:52.453487 16197 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34382fb050d946e68581751161ae3ecd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 10:16:52.662256 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:52.659212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.8.238.61:33359)
0811 10:16:52.659374 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.659376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.659463 (+ 87us) server_negotiation.cc:407] Connection header received
0811 10:16:52.659577 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.659578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.659656 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.659782 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:52.659960 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.660500 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.661353 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.661567 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.661651 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:52.661843 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:52.661987 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:52.662084 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:52.662109 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 10:16:52.662136 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260811 10:16:52.668366 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:52.665034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.8.238.60:43121)
0811 10:16:52.665218 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.665220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.665231 (+ 11us) server_negotiation.cc:407] Connection header received
0811 10:16:52.665331 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.665332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.665379 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.665469 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:52.665648 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.666212 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.667259 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.667516 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.667597 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:52.667954 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:52.668087 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:52.668163 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:52.668207 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 10:16:52.668233 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
I20260811 10:16:52.674548 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:52.670948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.8.238.61:33359)
0811 10:16:52.671012 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.671015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.671305 (+ 290us) server_negotiation.cc:407] Connection header received
0811 10:16:52.671392 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.671393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.671407 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.671504 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:52.671717 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.672585 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.673555 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.673965 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.674054 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:52.674231 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:52.674272 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:52.674322 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:52.674405 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 10:16:52.674435 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 10:16:52.674913 16375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:52.671665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49012 (local address 127.8.238.60:43121)
0811 10:16:52.671712 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.671714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.671941 (+ 227us) server_negotiation.cc:407] Connection header received
0811 10:16:52.672067 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.672068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.672078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.672132 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:52.672278 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.672987 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.673931 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.674263 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.674329 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:52.674491 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:52.674529 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:52.674557 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:52.674785 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 10:16:52.674819 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 10:16:52.675346 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 10:16:52.670958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.8.238.62:39425)
0811 10:16:52.671244 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 10:16:52.671247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 10:16:52.671536 (+ 289us) server_negotiation.cc:407] Connection header received
0811 10:16:52.671603 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 10:16:52.671604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 10:16:52.671672 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 10:16:52.671812 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 10:16:52.671931 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.672586 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.673654 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 10:16:52.674063 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 10:16:52.674128 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 10:16:52.674334 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 10:16:52.674461 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 10:16:52.674647 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 10:16:52.675246 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 10:16:52.675279 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":195,"threads_started":1}
I20260811 10:16:52.678047 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15777
I20260811 10:16:52.706025 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 15907
I20260811 10:16:52.722741 9144 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6YJKWE/build/release/bin/kudu with pid 16035
2026-08-11T10:16:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3363 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6687 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288673 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST